Google Voice Problem

[neil154]

I have just started using Google Voice in addition to my Verizon provided telephone number. I have Voice set to "ask me every time" and it always pops up and asks me if I want to use Voice or not, as expected. Some time when I tell it to use Voice, it tries but then says can't use Voice and gives me a choice to try again or call without Voice. If I try again it continues to fail, so I usually then tell it to dial without Voice. Then I can try to make another call and to use Voice and it works fine.

Does any one know what is causing this problem and how to avoid it?

PS: I have a Verizon S5 using Marshmallow