I like having a screen protector in place because particles of hard grit are everywhere. The earth's crust is composed largely of materials that are quite hard. That "dirt" gets ground into fine particles by natural erosion, human activity, etc., and is everywhere, including the air when there is wind, etc. No place is totally safe except, perhaps, a high level clean room.
Anyhow, you can pretty much count on particles of hard grit being in your pockets, purses, etc., at all times.
If the tip of your X-pen traps one of these particles between it and the phone's screen, even that will cause a scratch.
The fact that hard grit is everywhere is why you need to gently brush a lens off before cleaning it with a cloth or the like. Just wiping a dusty lens, or a phone screen, will push any hard grit particles present against, and then grind them across the surface, causing tiny scratches.
And if a larger particle of grit gets wiped against a lens or phone screen, it can leave a deep scratch.
Around here, it's often dry, and often windy. So folks here are all too familiar with grit being everywhere. But grit is ubiquitous everywhere except special clean-rooms.
So as much as I dislike the loss of sensitivity and the change in the look of the phone caused by most screen protectors, I live with it. And the same goes for using a protective case. The one I use is big, ugly, and totally destroys the look and feel of this beautiful phone, but I've had phones saved from extreme falls by good cases, so I put up with it.
Some phone maker should design a phone and case/screen protector at the same time such that the phone's look and feel are better with the case in place. The case should be part of the original design. They could sell these cases in different colors or feels, and the customer would end up with a better look and feel as well as a way to renew their screen surface and outer phone body in case of damage.
They spend a huge amount of effort to make the phone look great, feel good in the hand, and respond well to the touch screen, and then nobody uses the phone that way. That seems like a huge error in the overall design of pretty much every phone out there.