I have the same problem with my non-rooted, Verizon Galaxy Nexus.
When I attempt to power on after switching batteries, there is a 60-70% chance that I will get stuck on the screen that just says "Google", requiring me to take out the battery and try again. If the startup process makes it to the screen with the Nexus logo (four-colored X), everything proceeds normally.
This has been happening to me for several months, at least as far back as September. My phone did not always have this problem.
I have one standard battery and one extended battery, and this problem happens whenever I am switch from one to another (doesn't matter from which to which). I am not suggesting it has to do with the different battery capacities - just providing more info. I don't know whether powering off, removing, and then replacing the same battery would cause the problem too, because I normally switch batteries only when the existing one completely dies.
The only other piece of information I have is that the chance of freezing seems to go down if I wait 5-10 minutes before powering back on after putting in the new battery. I say "seems to" because I don't have any real data to back this up - it might just be in my head, seeing a pattern that isn't really there as a result of frustration from having to keep trying to power up over and over.