Experiencing the same problem - started happening a few weeks ago, not sure what changed. I have 4-5k contacts.
Tried all the examples listed here but none worked.
Inside the contacts app do the following:
Contacts->Settings->Only contacts with phones (checked)
Config info:
Verizon S4 Mini...