Check your card... It sounds to me like it's un-mounting itself and re-mounting, which causes the system to 'not find it'.
The issue is mainly with SanDisk cards, especially 32GB ones. Something to do with the card's chip and newest Samsung phones not being compatible. It un-mounts the card randomly, so when the phone tries to access the card it won't find it, and then your settings would default to internal memory again.
A good indication if this is happening to you (and you've missed the un-mount/re-mount messages popping up) is to check your battery stats; if Media Scanning is taking a larger chunk of your battery then that's your culprit. Actually, that process shouldn't even be in your list after a couple of hours at all unless you constantly swap cards after reboot.