Maybe I'm misunderstanding. By "speed dial," you're talking about pressing a number on the phone dialer and it dials the contact? If so, how about just adding the contact to one of your home pages and just touching the picture to dial? I believe you can do that by bringing up your full contacts list, bringing up the one you want to add, hitting the menu for that entry, and Place on Home Page. My wife's got the Nexus 5, so that's from memory.
Also, what about just saying "OK Google. Call Joe Bag o' Donuts?"