I had to use a separate app for this called Nav Launcher.
Although there are other things I wish I could do as well with Google Navigation, such as save my current location to favorites, select a detour route if I need to avoid a certain area for traffic, or just activate Navigation view without actually having a specific destination set.