Just got my N4 Friday and it won't connect to my wireless network. It detects the network and saves it, but it will not connect. I called the Google Play support line and went through all their troubleshooting steps, but still nothing. They're sending me a new device and paying the postage for the return. They said I can keep the old phone until the new one arrives. It sucks to have the issue, but their customer service is outstanding! Calls are answered quickly, they have good troubleshooting documentation and they're pleasant to deal with.
BTW, my router is a WRT54G with Tomato. I have paired other Android, Linux, Windows and iOS devices without issue, so I'm certain my setup was correct. Plus, they were pretty quick agreeing to send out a new device, so I'll bet they know there's a real issue with the N4 and wifi.