I have EXACTLY the some problem.
I tried a dozen things to no avail. Wiping the cache and factory reset seems to improve but the problem doesn't go away.
Mine is a Nexus 7 Wifi 2013, It happened after 5.1.1 OTA. NOW I'm on Android 6 and having the same problems.
It's making my tablet unusable...