The phones are identical other than screen resolution and battery size, so choose based on what form factor you feel will work better for you. If you take in more media during the day, like YouTube videos, spend a lot of time on the internet, Facebook, you may be better off with the XL as the bigger screen makes it easier to take all of that in. If you enjoy portability, using your phone with one hand, and not having to worry about where you're placing your phone when you're sitting down or whether or not it's going to fit in your pocket go with the smaller pixel. The great thing about these two phones is you're really not making a sacrifice going with one of the other.
I went with the smaller pixel, and even though the XL has a bigger battery and it's supposed to get better battery life, I'm finding the battery life on the smaller pixel to be very very good.