I have the Note 4 Innovator Edition. The fogging comes from the temperature difference between your phone and your head. And the longer you wear it, the more heat gets generated with your head.
I'd recommend to play in a cooler environment, play below a ceiling fan, or playing with a fan...