Having had my Nexus 6 a month now, after using the Nexus 5 for a year, I'll throw in my 2 cents.
The Nexus 5 was, and remains, a stellar phone and a far better value for the money than the Nexus 6. If a new Nexus had not been introduced, I would still be delighted by the 5 and in no rush to change (especially since Lollipop), though things like better camera and larger screens are still attractive.
The Nexus 6 hits a home run in screen size/quality and sound, making it, imo, the current king for media viewing - and I do a great deal of that on my phone. Battery life was never a big problem for me on the Nexus 5, but knowing that the for the few times the 5's battery was an issue for me (e.g., on a long day out during vacation where I was taking a lot of pictures/videos) the Nexus 6 would easily make it through is a huge benefit.
Camera is light years ahead of the Nexus 5. Simple fact is, Nexus 5 could take some very good pictures, but was not competitive at all with the best phone cameras out there. The Nexus 6 is.
The main thing you need to determine is if you do or don't have an issue with the phone being too big (you need to put your hands on one, or at least on an iPhone 6 Plus to get a very close approximation, but bare in mind that if it's not uncomfortable but still feels 'big,' you'd quickly grow accustomed to it - like driving an SUV for the first time). If that isn't a concern, and the cost of the device isn't an obstacle, I can't imagine any way in which you wouldn't be delighted in it as an upgrade in every way over the Nexus 5.