I have the same issue with Nexus 4 and ATT. Never was a problem with Galaxy Nexus.
Is this issue resolved with a new Nexus 4 or is this a problem with this model?
If a problem with this model, I'm going back to Galaxy Nexus, which is a much better phone only b/c of very poor connectivity issues...