I own both the one and the s4 and you can't go wrong with either, it just depends if your willing to deal with a worse camera and smaller battery to get a MUCH better screen and build materials so now slimy plastic. Also a much better UI that's always smooth. But both are a fantastic android experience it's just personal preference. Go to a local store and play with them side by side, it's the only way you'll be able to make the right decision.
For me I always go with my one, it's probably my favorite device I've ever owned and believe me I've had an extremely unhealthy amount of phones lol
-Verizon HTC ONE-