I haven't bothered trying to recreate it, but the one time I had it happen was when I put the main screen to sleep with media controls on the second screen. Phone went in my pocket like that, and when I pulled it out to wake up the second screen I had the same issue as you.
It was persistent...