This is strange. I don't even have the oem charger any more. I use a 1 amp HTC charger from my EVO, a 1 amp palm charger from my Treo Pro, and a .8 amp blackberry charger. All work in GB or ICS. I'm on the stock 4.0.4 right now and have never seen this problem.
Did this start happening the moment you went to 4.0.4? How did you get to ICS? Did you OTA, fastboot, etc.
It's a long shot but perhaps a factory reset might fix it?