This finally worked for me, after failing 50 times. Someone suggested saying "let's go to "name of contact"'s place" . I tried a few times and it didn't work until I said the exact full name of the contact. Then it asked me which address . I said home and it worked. I'm going to try more...