I had this same problem! i had over 1000+ contacts, but actually only under 200 in phonenumbers and emails.
I actually used this program called "contact remover" which is much easier than going through the phonebook on-by-one deleting multiple entries. Think of it as a batch delete. I think the paid version deletes duplicates for you.