I already have a Nexus 6 on preorder but I went to Best Buy to handle a Note 4 because every review is saying 'it's so big!' (we know that, yes). It's big, but I'm willing to give the Nexus 6 a chance.
My bigger concern more than using it and holding it is how to carry it around. I don't wear skinny jeans, but I wear slim-fitting pants, and one review mentioned that sitting it in with a front pocket was uncomfortable. May have to just take it out every time I sit down. Just going to have see how it is.