Enable everything under Location Access, also get this app and see that you're actually getting satellites. Your location is probably off because it's taking location information from the cell tower or your WiFi, though 6km is a huge margin...
Bottom line: GPS needs to triangulate your...