I have the same question. I find no way to change the audio output destination in any of my apps, but primarily Google Play Music and Pocket Cast unless it's to a Chromecast. I could've sworn it existed in the past but can't find the option anywhere. To clarify, for example, in those same apps in iOS, if I touch the AirPlay icon, I can select from the phone speaker, any AirPlay compatible device in range, or any connected bluetooth device. This is in addition to the Chromecast option. On Android--nothing like it. If I want to switch from a bluetooth speaker to the phone or vice-versa, there appears to be no simple way to do this. This seems bewildering to me.
I hope I'm just being daft and missing it. Appreciate insight from anyone that can help. BTW, I'm presently using a Pixel XL if that matters for any reason.