You can also go to your dialer, dial *#*#4636#*#* third option that pops up is Battery Information. That will tell you your actual temp. I'm not sure the exact temps, but mine running fairly hard (streaming youtube over 4g, music and a game, etc) and charging, will get close to 50C. 40C is easy to get to just surfing, and will feel warm to the hand. Both are fine temps for what is going on. Over 55C you should think about stopping what you are doing.
I had a phone that was overheating, not sure how hot it would get, but it was so hot the battery would drain, while charging, while I would just surf the net at a rate of about 50% every 20-30 minutes. It got traded for a new one, all has been good.