I spent considerable time using the galaxy s4 as a daily driver. With the features I had enabled, and the apps I used, I noticed significant lag performing sometimes the most simple of tasks. Unresponsive screen while trying to answer a call, app transitions, etc... I was rather disappointed. However running the Google edition ROM on the device made it significantly more responsive mostly due to the absents of touchwiz. Basically the majority of the features of the s4 lag the device.
Jumping over to the HTC one, I noticed pretty much zero shutters and minimal lag. App transition was smooth, gaming, etc. I decided to adopt the HTC one as my daily driver. Its a top of the line smartphone that isn't over saturated with useless (in my opinion) features similar to those found on touchwiz. Its snappy out of the box, as it should be.
A month later I'm still using the htc one as my daily driver and couldn't be more happy. Add the custom sense ROM and mods, it makes it an even more enjoyable device to use and own.
In the end like I have said before it's all personal opinion. However its rather unfortunate a quad core phone would be unresponsive at times and just generally laggy. Now I've heard others whom say they have zero lag on the s4, and that all comes down to the variations in use.
I really wanted to love and keep the s4, I just couldn't see myself using it for one to Two years. Others love it.
HTC One - Unlocked, ARHD 12, Bulletproof Kernel, Several mods.
Nexus 7 - Cyanogenmod
Tab 2 7 - Cyanogenmod