Honestly I can't imagine you going wrong with either, though I've as yet only tried the N5. How much do full voice control and active notifications matter to you, I think those are the two biggest questions. I was once worried about N5 battery life--and while the Xs battery may be better still--after having the nexus a short while now I no longer fear not having a charge to last me through the day with mixed use.
Posted from my N5 via the Android Central App