Anyone using Tmobile note 4 ever notice that the s-voice sometimes starts on it's own!? I m sometimes playing a game or just browsing through FB etc an bam it activates by itself as if there's a ghost in the darn fone!!
I'm not familiar with S Voice, but is there a hot word that can initiate it? If so, then perhaps it's picking up sounds in the background. If not, and it's really initiating on its own, go into system settings - applications - Sort by all - s voice. Clear cache and see if that fixes your problem.