Yeah, you should...if it's configured that way. Make sure your e-mail app or the gMail app is configured to make a sound when you get an e-mail and also make sure that your phone's sound profile is not set to mute notification sounds (i.e. in Vibrate mode or in Silent mode).
If you're not getting the notification icon then I'm guessing you're NOT getting the e-mail notification at all in your phone. This could be due to the Sync option being off or if you turned off data syncing for that specific account in the Settings/Accounts section.