The music app I'am using which is new to me is PowerAmp, and yes it is set by default to start playing where it left off when you plug headphones or anything into the headphone jack. You can change it by opening the app, typing the ... in the bottom right, tap headset/bluetooth, and uncheck...