The size is a funny thing. Me, personally, I got huge hands, so I can one-hand the thing for 95% of the time. But I can see how someone coming from a smaller phone or has small hands, would kind of have issues. That being said, there are advantages to the size. Watching video on the thing is a dream. Between that massive screen and the front facing speakers, watching a movie on phone is quite enjoyable.
As far as needing the absolute latest phone... This was a funny generation for phones. The last of the previous generation... The Nexus 6 and Note 4, the ones powered by the Snapdragon 805 chip, really don't give up all that much to the newer phones. In terms of pure power, phones like the S6, Note 5 and 6P are faster, but the 805 is no slouch and actually can outperform the newer chips under extended heavy loads because the 805 doesn't have to throttle nearly as quickly or as much (to reduce heat) the new 810 and Exynos chips.
When looking at things head to head with the 6P.... all things considered, I am really only gaining a better camera and a FPS. Which isn't trivial, but it's a short list. The screen may be better, but I have no qualms with the Nexus 6 screen... the 6P is faster, but its like saying a 210 mph car is faster than a 200mph car... the latter still goes 200. And everything else is kind of a wash since functionally speaking the 6 and 6P run identical versions of Android... (unlike, say, Touchwiz, where two phones running the same version will look and feel different).
I'm still considering getting a 6P, but I'm still hemming and hawing far more than I expected.... and that is 100% because the Nexus 6 is so awesome.