make sure you are not hitting it when you answer your phone.
If you are not selecting it,it sounds like a software glitch. The easiest way to solve this problem is just to hard reset your device. If your contacts are not backed up to your google account, back them up onto your sd card.
hope this helps.