if you're signed into Latitude on the device, it will ping your location in the background quite often even if you're not tapping through the Latitude app.
sign out of Latitude (actually tapping "sign out", backing out of the app doesn't suffice) & see if it still comes on.
when the location icon appears, even if you aren't running an app in the forefront of the phone, check your "running services" section to see what GPS type of application (maps, Places, four square, latitude, internet browser) is running to see which is using it.
also, are any of your Location Services check marked (Google Location Services, Standalone GPS Services, Phone Finder, VZW Location Services) ?