I'm hoping Google can address the lag issue in future updates.
It is an issue.
It doesn't mean it's an issue for all or even most 5x users, but the fact that there are many users complaining about it (not just one or two outliers) means it's an issue.
Saying "Don't use certain apps" or "Wipe the system cache" is a workaround, not a solution.
I've experienced lag on the Nexus 5x, and I believe it can be addressed with a software update. My old Moto X 2013 running Lollipop had zero lag. The Nexus 5x (without screen protector, without restoring apps, about several hours after wiping the system cache) has lag. It isn't dealbreaker lag. It isn't horrible lag. But it's lag, more lag than should be on a newly released device. Most of the lag for me has been in switching or first-time launching applications. Within apps, not so much lag.
I'm still trucking on, though. Love this phone for its fingerprint sensor and it's amazing camera. Hoping the lag will go away in a couple more updates.