Ok Google won't pick up contact name

[AC Question]

For some reason when I use "Ok Google" on my LG Urbane, and have it send a text to certain people, it just keeps saying "to whom?" And no matter how many times I say the name, and despite it picking up the name, it keeps saying that. Oddly enough, it sends texts to certain people, just not others. Tried changing the contact name but that didn't help. Any ideas?