Your friends must have extremely good hearing.
I on the other hand, have something like 20 dB loss on good days. Tinnitus ringing in my ears all the time masks what folks are trying to tell me.
Sometimes the speaker phone option works well for me, other times I can't hear it either.
I usually use a BT headset for truckers, a Blue Parrott which has very good speaker audio. And for you, it would offer very good privacy.
* Blue Parrott VXi Corp :: B250-XT+ :: The best just got better. Like its predecessor, the BlueParrott B250-XT+
I also have a Plantronics Discovery 925 BT earpiece that offers very good privacy, although the receive audio does not have the quality offered by the Blue Parrott BT250-XT+. It is an "in the ear" BT earpiece.