Are you able to turn it back to full volume? Like you, I also use the headphone jack a lot to plug into my car stereo and I know Android started an annoying "we know what's best for you" nag where it'll reduce the headphone volume when you plug anything into it. Rumor has it that it has to do with some European regulation that made them do this, and it's to protect your ears. On every other phone I've had, I could turn the volume back up if I cleared the nag message. On one, it was tied to its own app that I could freeze once rooted. I'm still working on getting a G4, so I don't know if there is a solution to it on this phone.
But I agree, it sucks. I listen mostly to podcasts, which seem to be notorious for being encoded at a low volume. Even with the phone volume at max, I sometimes I have to nearly max out the stereo volume to hear them clearly.