I love my Nexus 4, and have had it since the second batch shipped out, but I am starting to grow a bit weary of the issues it has. I think I will wait a bit longer, and perhaps for the Note 3, until I switch.
Pure Google is wonderful, but as I'm sure many of you have noticed, Google is trying to run around the OS fragmentation by releasing many of the "Nexus" applications/updates on the play store. That alone makes me want to switch. I love the pure Google experience, but it's lacking in many of the interesting features other custom Android OS's offer (yes, as crazy as it is to say, the custom operating systems do offer some cool features). I know my other option is to root and possibly upload a custom OS, but I really don't have that time in my hands right now.
Good choice by the way, and have fun with it!