I'd definitely have to go with losing the trackball. I do use it more than I thought it would, but it doesn't seem as smooth to me as a blackberry trackball (I liked how you could change the sensitivity of the BB trackball). Also, I don't really seem to scroll too fast/high when I'm in portrait mode, but when I'm in landscape mode, I'm forever scrolling too far to the left and hitting the home soft key. It's by no means a deal breaker, just annoying, but still.
But I agree with Jeremy - out of the phones I've had, I'd still pick Nexus every time. I don't have a problem with the screen - I like how my widget can make it different brightness levels - I just put it on the higher brightness setting when I"m outside. Maybe this will make a bigger difference when it's summer and I"m outside all the time, but for now, it's been perfect. The screen is amazing. I use Tweetcaster for Twitter, and the screen in the dark theme is absolutely beautiful - I'm amazed every time I see it!
I also kind of wish the camera didn't stick out like it does. It's not bad, it doesn't interfere with anything, it's more because I"m afraid of scratching it when I set it down, and I don't like to set my phone face down.