I have the same problem when connected to the MP3 input in my car. The iPhone and X worked fine like that. I could hear the caller over the car speakers and they could hear me through the phone mic with it in the center console. Worked great as a hands free. There should be a setting to enable...