The majority of negative issues related to Android updates can be fixed with a simple data wipe. It's ALWAYS best to do a factory reset when moving from one Android version to a newer one, but the OTA update packages don't do this for you because it would scare users. But when people complain about the updates, the first thing a CSR will tell them is to wipe their data.
So make sure your contacts are synched to Google, your pictures and music and all is on the sd so you'll be fine. For your apps, use a backup program, let Google try and re-sync everything, or just remember what you want and re-download them. I can assure you from first-hand experience that this update does NOT have issues, and will improve your phone's performance and battery life noticeably.
If you simply don't like the added features or tweaked interface of ICS, I don't know what to tell you. Most everyone loves that, but you can use it pretty much like you used GB and skip using the new features, though the task switcher is the single best part of any modern mobile OS, imho.
Hold out if you want to, but it doesn't make sense to go through all that hassle to prevent your phone from getting ICS, this coming from someone who's gone through alot of trouble to have the latest builds of ICS on this phone for several months now. Accept the OTA when it comes, and wipe your data like I said, and I think you'll really grow to love it.