This may be a coincidence, but a couple of days ago I tried turning off location history. Still have location itself on all the time on high accuracy. Since I turned off the history, that icon seems to only pop up when the phone actually determines my location. And it does not stick around. So with location history off it seems to be working in a more logical manner.
By logical here is an example: If I start Google Maps, the icon pops up. If I pull down the notification menu, the icon message is "Finding Location", then quickly changes to "Location Found". The little notification icon stays up there as long as I am using maps. As soon as I close out of maps, the icon instantly also goes away.