Raf,
Do you have any 3rd party 'phone' apps (ex. Skype) installed?
Since it's only (?) happening with the Phone app, you could try this before performing a Factory Reset - go to Settings -> Application manager -> swipe right until you get to the ALL tab -> scroll down until you find the Phone app, and select it -> select the 'Clear cache' and 'Clear data' buttons.
If that doesn't help, go to Settings -> Application manager, press the Menu button and select 'Reset app preferences' (note: this will clear all of your default app selections).
BTW, does this only happen when you receive a call or also when you attempt to make a call; or does it happen as soon as you open the Phone dialer?
Sent from my rooted, debloated, deodexed Sinclair ZX-80 running CM 10.2 using Tapatalk 4