Do wired or Bluetooth headsets work? If so, then the earpiece speaker might have failed.
Is it completely silent over the earpiece speaker and headset, or is it just really low? Did you just get a system update?
This is obviously very basic, but I still should ask: the phone volume is turned all the way up during the call, correct?
Try toggling Verizon's Advanced Calling (their voice over LTE service) off or on.
Since you also have the same problem when using headsets, it's clearly not just an issue with the earpiece speaker. Try a cache partition wipe, and if that doesn't work, try Safe Mode: http://support.t-mobile.com/docs/DOC-10404. If it persists, contact Verizon or Samsung about this.
Hope it works! But I'm a bit confused, now--you're mentioning the microphone, but I thought it was a problem with hearing the caller over the earpiece or headphones. Was the problem actually that the person you were calling couldn't hear you unless you used speakerphone?
The primary mic (near your mouth) handles normal voice calls while the secondary mic (at the top of the phone) usually provides a noise-cancelling function. But if you use speakerphone, then in most cases, the secondary mic is used as a regular mic (either in conjunction with the primary mic, or alone). This is why people can hear you on speakerphone if your primary mic is faulty.
So just to be clear--is the problem that (a) you can't hear the other person speaking to you over the earpiece mic or headphones, and can only hear them if you're on speakerphone, or (b) the other person can't hear you if you're not using speakerphone?
