I've had both for a month. I think I'm taking the HTC One in the end. Several things did it for me:
-The One has a better screen, bar none. Much brighter, better colors, much whiter whites, sharper. The S4 does have good blacks though, and I like that it's 5". Still, hold them both side by side and the result is clear.
-The One is noticeably smoother, and the interface is much slicker and more refined (imo). I like some things about Touchwiz (quick settings, customization).
-Internal storage (25.x gigs) is nice, more so than SD storage. Faster, much more reliable, eliminates a common source of errors. Samsung's upcoming Apps to SD implementation is far, far from perfect.
,
-The One's front-facing speakers make a difference. But in terms of actual speaker loudness, the One wins; but not by as much as you'd think.
-Battery life is about equal on both depending on what you use em for.
-The S4 is sturdy and feels good in the hand. I like that it's thin and light. But I still much prefer the HTC One in terms of materials, construction, feel and looks. It's really no contest for me here either.
-The S4 has the better camera. I like the One's low-light and that it snaps pics much quicker though.
-I will miss the S4's CRAZY fast charge time, customizable notification light with lightflow too.
It's a tough choice. Unfortunately not everybody gets the luxury I had. Thankfully I was able to pick the One (see what i did thar?) that best suited my needs.