OK Google & Directions To Saved Contacts

Just wondering if anyone else has figured out how to use "OK Google" to get directions to addresses of contacts saved in to your phone. Seems like something they'd definitely have built in to the software, but I'm having no luck at all in enabling it so far. Basically I'm saying "OK Google: driving directions to John Doe" and instead of using the address associated with the contact John Doe and loading up directions with my GPS app, it does a Google search for "driving directions to John Doe". Whereas if I say something more generic like "OK Google: driving directions to AT&T Park", it fires up my GPS and has directions loaded and ready.

I say "show me John doe's address" then tap on the address and navigate to it. I also can't get "navigate to" to work.

Sent from my Nexus 5 using Tapatalk

Yeah. I believe that's the best I could come up with as well. Would be really handy to do the whole process hands free with one simple sentence.

Internet blogs indicate not possible YET

Can be done with MAPS / Directions.
Voice request brings up contacts entry for selection.