I'm new here, I actually use a BlackBerry but I also have a G2 but I think I can help since this happened to me 2 days ago.
If you are using Google Voice switch to the carrier voicemail (settings - call settings - voicemail service) then call your phone from another phone and leave a message, listen to your message, then delete your message, your icon should go away. Then change back to Google Voice.