It really bugged me at first, there are already enough things that clog up my notification bar just to turn the phone on. I found that it appears then fades away so it has bugged me less. It is there on the lock screen always, I use a PIN. Once I log in, it dissapears after 1 - 2 seconds. I usually only see it reappear when I pull down the notification bar, but again it goes away when I close it.
At least 4.0.4 is better with silent/vibrate than 2.3.6, that blasted OS would show a silent and a vibrate icon when I turned that on... waste of real estate! 8)