I've had the Z3, the Z3 Compact and now the Z5 Compact. As far as Nexus device go, I've had every single one of them. Now onto your questions.
Size:
The N5X looks and feels ginormous compared to the Z5C. I was a bit amazed last week when a friend of mine, who has the Note 5, compared it to my Nexus 5X. They are almost the same size if that gives you an idea about the N5X size. It's pretty large for only having a 5.2" screen.
Screen:
My Nexus 5X has warmer color on the screen than my Z5C does. Whites look a tad bit pinkish on the N5X which I personally do not like. The screen on the Z5C looks better to me but I guess that's going to vary person by person. The N5X has the better spec'd screen but I prefer the Z5C screen over it.
Speed:
My N5X lags a little in certain areas. I have not noticed any lag on the Z5C. I did get a Sony firmware update right out of the box so I can't compare it to previous Z5C software. My Z5C screams! It's fast! Is it faster than the N5? A little in some areas but nothing major in my opinion. As I previously stated, I ditched the Sony launcher in favor of the Google Now launcher.
Feel in hand:
One handed use on the N5X is doable but depending on your hand size may not be possible. The N5X feels fine in hand. I've read many of the so called expert reviews where it's called cheap. I did not get that feeling when holding it. Does it feel 'slip out of your hand premium'? No, but I have no complaints with the feel in hand with the N5X. I am only on day one with the Z5C but for one handed use it rocks! It does feel more premium in hand -v- the N5X but again that is going to vary person by person.
The N5X is a fantastic device but it's larger than I wanted. The Nexus 5 was almost the perfect size for me and the 5X is a little to large for my liking. I am only on day 1 with the Z5C but if cash is not a consideration, I'd recommend it over the 5X...