Why does google voice dialer only recognize some of my contacts?

[AC Question]

I have 2 Moto G plus phones and both are experiencing the same problem. Some contacts can be voice commanded with a bluetooth device that is using Google voice dialer and some are not found even though they are clearly part of my contacts on the phone app and are also on my People app, Nobody at the stores that I bought these devices have any real answer. What can be done?