I posted a thread about this a few months ago. It seemed dumb that it took so long to load text messages. Nobody was able to give a solution, I tried removing some of my old contacts and got my number of contacts down to like 85, and it didn't help at all.
I installed a ROM and the problem was fixed. I didn't experience this issue on PACman, Viper, or my current ROM CM10.1