I have two gnex phones and one got it right after I did the clear framework trick. The other one, I cleared the framework three times and it got it on the third. Could just be bias, but it certainly wasn't working on either of them until I did that.
And I don't know about the diffs from 4.1 to 4.1.1 because I didn't try 4.1. However, the different from 4.0.4 is a good bit. The menu is definitely noticeably smoother. I compared using my two before I updated the second. Also, Google Now is just awesome. I haven't figured out all the useful commands, but it responds to a lot and only gives me typos on ambiguous words.
Although, I was excited to once again have the microbes live wallpaper when I switched from SGSII to Gnex, and now I've lost it once again. :'( Now is worth it though. I used voice controls quite a bit before, but it is nearly flawless now and I find myself using that and the notifications from now quite a bit.