Yep, I went and checked out the S4 yesterday and will probably do so again today. I really like the One and it's the best phone I've had to date. My two main reasons for having any doubts...
1. Camera. I'm a camera junkie and the S4 has a great camera. The One is the best camera phone I've had yet, but I still can't past the S4. What I'm trying to decide is how important that is to me and the types of conditions I'll see the benefit in. The One is no slouch, but you can definitely see the difference in pixel count. The S4 does take good low light pictures... maybe not as good as the One, but with the in store conditions (pointing to a dark area) it was right up there with the One. In medium to brighter light, the detail is definitely better than the One and the S4 has better metering and exposure. I know the One is scheduled for a camera app update, but I can't judge that right now.
2. The LED notifications. I'm pretty disappointed with the LED notifications on the One. Might not seem like a big deal for some, but not having any notification status for apps while charging is a huge oversight. I keep my phone on a dock charging at work and being able to glance over to see notifications is important. I have to keep it on silent, so I can't use sound notifications. I don't mind the simple colors (just two), but the lack of control or features is really nagging.
What's really preventing me from getting the S4..
I like the feel and build of the One much more. I love the front facing speakers. While the S4 display is bigger and I prefer it's size, I think the color representation on the One is much better. On anything but the brightest setting the appeal of the S4 quickly drops off. I really dislike TouchWiz. It's a mess, regardless of capability. I would install Nova right away, but I still don't think it would get rid of enough of it for me. Too many gimmicks and "features" on the S4 I'll never use -- I'd wind up disabling 3/4 of them. I also dislike the stock email and calendar apps, yes, I know I can get others, but the OEM/stock versions on the One are really good, in fact better than any 3rd party I've tried. Also, I want 32GB internal storage which is unavailable during my return window -- I don't care what I can get on SD expansion.
Boils down to the camera and notifications really and if those are enough to overcome what I dislike about the rest of the S4. I don't think they will. Gut feeling, I'm sticking with the One, but still a little torn.