For whatever reason, every time location is toggled off and then back on, it goes to the high accuracy setting by default.
My personal experience with location services is that GPS only is the best setting. Battery saver or low power cause more trouble than they are worth in my experience.
I use Access Lock to remove location permission (along with keep awake) from just about every app that I can (it's limited to downloaded apps).
My theory, whether it's accurate or not I'm not sure, is that with location turned off, something is trying to access location but it can't. And for whatever reason, it keeps trying over and over again which is why GPS use can be high even with location off.
I've used this setup (access lock with location permission revoked plus GPS only) for my last 3 phones (G3, G4, V10) and it's worked flawlessly.