I've had this same problem. It's not a cable/charging problem since the phone will charge when powered off. In fact, I can plug in the charger when the phone is on, see that it's not charging, power the phone off, and see that it starts charging. Same charger and I didn't even unconnect the cable. I had this problem for a couple weeks, then it went away (phone charged like it was supposed to). Then, it came back again two days ago. It would only charge when I powered-down the phone.
Then, today, I was experimenting and pulled the USB cable out of the little block that plugs into the wall, and I stuck the USB into my laptop (phone connected and powered-on the whole time). The laptop went crazy with repeated messages about an invalid USB device. I unplugged it, waited a few seconds, and plugged it back into the laptop again. The Galaxy Nexus was recognized by my laptop this time like it's supposed to be. Plus, it was charging! When I then unplugged the USB from the laptop and re-plugged it back into the little wall-brick, the phone recognized that the charger was connected and it started charging.
There is NO WAY that this problem is not a software issue. Hardware problems don't manifest themselves like this.