I bet it's internal clock drift (since the phone is often battery powered and subject to varied temperature its quartz clock can be less accurate) but the network sync is getting interfered with so it becomes more pronounced. There was a "fix" in the latest SC rom but i havent seen a description of what it does, although i would wager its some way to force a network sync to happen more often.
This "lost time" happened to me once, just as others described, losing time overnight to the tune of 5 minutes. I fixed it by turning off wifi, and for some reason the phone immediately did a network sync and it went to the right time. Since then i have been less apt to leave wifi turned on when im not using it, and it hasnt recurred since.
Remember, 5 minutes is about 0.34% of error over 24 hours so it doesn't take much of a drift rate to see numbers like this.