Definitely something to do with the SD card. After doing all of that yesterday, it still got stuck on 5 photos. So this morning I shut down the phone, pulled the SD card out again, booted it up and let Google Photos run. And within seconds, it said back up complete.
I then shut down, and...