Well, it all boils down to how you use the thing... As a productivity tool, the Note 4 is still pretty much king. If I was a lawyer, doctor, clerk, someone who takes a lot of notes and the like. The Nexus on the other hand is more like a media consumption device. Large screen, loud stereo speakers that flank you if you hold it sideways. The Note 4 is a small tablet and the Nexus 6 is a large phone. They are the same size, just an entirely different approach.
Now, personally, I've had some not-so brief moments behind the wheel of a Note 4 and its nice and all, just dated and drab. I hate Samsung's strange obsession with greens and muddy blues.... It's like the Little Mermaid puked all over the phone. I don't think I could stomach looking at it the entire time. But that's just me.... the Note 4 is still a top notch device, but so is the Nexus 6. Again, it depends on what you want to get out of it.