Go to contacts then go to the contact that you want to delete the history of. Click the contact so that it shows you their details (phone numbers and emails, etc.) and then click the circle with the down arrow in it next to the words "Contact History", select clear and then clear again.