I have the same issue and it does not matter if I restart the phone or not. I am also using an app killer to kill all running apps. Killing apps does not fix the issue either. I upgraded to ICS a long time ago, but have only recently gotten this problem. The only thing I have not done is a reset to default settings.