It wasn't long ago at all... 14 months or so... That I was using a 2.6" screen and that having stepped down from 3.1" (Palm Pre 2 down in size to HP Veer). Why did I go smaller? Upgrade in hardware. And at the time I adamantly said, "Screen size does not matter and bigger does not equal better."
Then I got a Galaxy Nexus (4.65") and quickly after a Nexus 4 (4.7"). Big increase in screen size. Why? Hardware and OS. I wanted to try some OS ports like OpenwebOS and Ubuntu Touch. The Veer fell to the wayside just to save from juggling too many phones.
Next was a Lumia 521. Got it on a deal as I was Windows Phone curious. I liked it. Had a 4.0" screen. Now when I stepped down in screen size I noticed it. Loosing 0.7" on a daily driver meant a perceptible difference. Going 3.1 to 2.7 wasn't as bad and it was the same amount of 0.7". There seems to be a threshold of... expectation that, once crossed, becomes a baseline. It is most likely a combination of size and pixel density. It is also something that very likely differs from individual to individual... Exactly where that line is. That point of no return where now, suddenly, anything smaller seems weaker and less satisfactory to your needs. I rapidly stepped up from the Lumia 521 (4" screen remember?) to a Lumia 925 with a 4.5" screen and sighed in relief. Part of that was going from a budget phone to one worthy of daily use at a higher level, but part of it was that extra screen real estate.
Now I'm driving a Lumia 1520.3 every day. 16:9 ration, 1080p @ 1080 x 1920 oixels, 367ppi, IPS LCD on a 6.0" diagonal. She's beautiful. A huge bump up in screen size of 1.5" gained. Could I go smaller now? Maybe... But not very. I've crossed another threshold; found a new standard of what is acceptable for my daily use. Did a 6" screen drive my choice. No. I actually thought of it as a hindrance. Hardware drove my choice. Double the RAM, big step up in CPU/GPU to the Snapdragon 800, 20MP camera, microSD slot, huge 3400 mAH battery w/ long run times, built in Qi charging. But I also love the screen and find the size not as unwieldy as I first thought it would be...
Do some people buy large screen phones just because of screen size? Undoubtedly.
But it probably is not as many as those who DON'T buy a phone due to screen size.
For myself, and I highly suspect a majority of iPhone 6 Plus buyers, it's just a quest for the best Hardware/Software bang for the buck and UX they can get; screen size being just one consideration among many and not necessarily near the top of the priority list.