I noticed this as well. Using Samsung's music player or Pocket Casts if I start audio with the screen on, then turn off the screen, the audio is switched to the speaker. I found what works to fix the problem is to queue up what you want to listen to, kill the screen, then use the inline remote on the earbuds to start up the music/podcast and it will play though the earbuds/headphones. I don't know of a solution that doesn't require the inline remote.
Oddly however, Pandora does not display this issue. You can start Pandora with headphones and kill the screen and it will continue to play as you would expect. Not a programmer so I can't speculate what is going on but there is an issue here.
Is there somewhere to submit bugs to Samsung?