This is usually an indication of a bad USB cable, so trying a new one would be my first suggestion. It could also be a bad connection in one of the USB ports. I doubt it's the battery itself, unless you're having additional problems like excessive heat or draining very quick.
That means that the phone is drawing more from the battery than from the charger. It could be a bad cable, but it's usually a sign of a bad battery. (Also make sure you're using the charger, not charging it from a USB 2.1 PC port.)