To be fair, absolute build quality of the One remains to be seen as any problems will take a while to surface, but I can't imagine there being any real issues. In terms of materials used, if you are fed up of your current plastic fantastic then you will love the One, the aluminium and glass gives a reassuringly quality feel and it really is a crime to put this phone in a case.
For me, that quality of material and ergonomics, just the way it feels in the hand, is a much higher priority than a 13mp camera or an 8 core cpu. Think about how you use your phone on a daily basis, you might only use many of the features occasionally but every time you touch the phone, even to check the time, you are interacting with it's ergonomics. The Galaxy S2 had the ergonomics (if not the materials) and even with it's relatively poorer specs it was fast enough to do everything it needed to do, the S3 sacrificed that usability for more (unnecessary) power and was a worse phone for it. I am not at all hopeful for the S4.