Welcome to Android Central! Does this happen with different headphones? What if you use headphones that don't have an integrated mic or remote control buttons?
Audible is freaking annoying. I love the service and the app but it takes over auto play. Even if I swipe it away it still auto plays, I have too force stop it.