I found that going into
Settings
Apps
All
Contact Storage
Then chose Clear Data......it wiped all the contacts from my phone then re-synced them with my gmail account version almost instantly
I'd make sure you have a backup or they are all in Gmail first before proceeding - cos if you lose...