If it's intermittent when you wiggle the end at the phone, the port in the phone might be loose. They should replace the phone for that (unless your carrier does real board-level repairs). If it's intermittent at the computer end, try a different USB port.
In either case, it could also be a bad cable. (And, no, for file transfer - I'm assuming that you have the phone set for file transfer or MTP when this happens - you don't need the driver - the phone acts as a thumb drive, and the computer can handle that without a driver. If you're trying to run adb you'd need a driver, but it would either work or not, it wouldn't be intermittent.)