Bluetooth headset sometimes gets directed through speakerphone?

[ykhan67]

Sometimes, my phone will play through the phone's loudspeaker rather than my BT headset. For example, I'd place a phone call through Google Talk through my headset, and despite being just placed through the BT, it will play the conversation through the loudspeakers instead. Google Talk will also usually say "Calling <person> through speakerphone". How do you prevent it from going through the speakerphone by default?

ZTE Grand X 4 (Z954)
Android 6.0.1