I am kind of regretting not returning mine at this point. It lags all the time. Just this morning when on a call I took it away from my face and it wouldn't turn the screen on. Answering calls is an adventure as it doesn't always respond to the answer swipe. I'm always double tapping stuff because I'm not sure if it has responded or not. Many apps stutter badly when scrolling. This is a disappointing phone, the best hardware coupled with terrible software. Even my original Surface RT puts this phone to shame in terms of smoothness. Not in speed though of course. This phone is fast once it does respond.
Some people on here claim to never have any lag. But look at reviews and comments online, there are still issues with TouchWiz and you don't see those issues with Sense on M8 reviews and forums. .
Now, my issues may have to do with what Verizon has done to it since they loaded it with bloatware and branding crap which can't all be removed, so I can't speak for S5s on other carriers. Regardless, there is no excuse for Samsung allowing this phone to reach consumers. This kind of thing was acceptable and even expected on Android 2 years ago. Not today. Every other flagship phone offers a more fluid user experience. Even the MotoX, which has been much maligned for its supposedly poor specs.
Go play with one in the store and see if the stuttering and lack of responsiveness bother you. You may not notice it or find it as annoying as I do. And as you can see, many people don't have any issues. But the problems are real and in my opinion you really need to love the features the S5 offers to put up with it. I thought I did, but the more I use it the less I like it.
On the flip side, I don't know if I could live with the M8's camera. My S5 has replaced my point-and-shoot. There are several things I don't like about the M8, but the camera was the dealbreaker.
As for some of the other issues you do have to keep in mind that Samsung sells a ton more of these than HTC sells M8s. There's going to be a lot more issues reported for that reason alone. I haven't had any glitches other than the responsiveness. In fact, it has been rock solid in that stuff doesn't crash and I haven't had to unexpectedly reboot it. Too early to tell anything about durability.