There's a difference between the phone getting your location (if you do a search, you might want it to be limited to your area. It's nice to know about a tire shop on another continent, but you need one nearby, so you can get your flat fixed), and reporting them back to Google, which not all apps do.
Maps, when it starts up, if you have GPS enabled, uses GPS to find out where it is. If you haven't run it in a few days, it needs to locate the satellites that are supposed to be currently overhead. Again, if you have "don't report this" set, it only stores locally.
Sure, the police can get a warrant for your phone, but just grabbing it to see where it's been without a warrant, in the US, is a violation of the 4th amendment, and all subsequent information coming from that knowledge gets tossed (theoretically).
It's like wanting the convenience of using a credit card without leaving a financial (and location) trail - either you lie on the grid or off it - you can't have the convenience of living on it and the privacy of living off it.