My theory: it's a design flaw in the way that volume + track control work through the headphone jack.
My S5 came with a set of headphones that have an integrated control for volume + track control, and I do not seem to have any issues with volume adjusting itself when I use those headphones...