OK, so, my situation seems to be better. So far, at least.
AT&T Note 4. No SD card. The SIM that AT&T sent with it. Slim plastic case.
GPS issues since day one on KitKat. Stock settings, GPS would "stop" in one place for 10-15 seconds, then go to "signal lost" with zero satellites in sight for 10-15 seconds, then pop back into place. This would happen once every 10 minutes or so.
Turning off "Enhanced LTE", GPS would "stop" for 10-15 seconds then either "run to catch up" or pop back into place.
Lollipop upgrade seemed to have little to no effect. Multiple erases of cache partition, pulling battery and SIM for a half hour, and even downloading a GPS toolkit that cleared the AGPS data cache, little to no effect.
Two factory resets (because if I'm going to go through the pain twice is nice), the GPS sucked the evening of the day I did it and..
..has not had a problem since.
Admittedly, that's a sum total of about an hour and a half using it, but in my case (knock on wood) a factory reset in Lollipop plus giving the GPS a little time to "figure things out" ??? seems to have cleared up the problem.