Hello AustinHans
This seems to be a Lollipop problem afflicting a number of different devices including LG, Samsung and HTC phones. Hopefully it will be fixed in Lollipop 5.1 or M - whichever is released first.
What app are you using for music?
Poweramp has a number of settings which may...