The primary mic (near your mouth) handles normal voice calls while the secondary mic (at the top of the phone) usually provides a noise-cancelling function. But if you use speakerphone, then in most cases, the secondary mic is used as a regular mic (either in conjunction with the primary mic, or alone). This is why people can hear you on speakerphone if your primary mic is faulty.
So just to be clear--is the problem that (a) you can't hear the other person speaking to you over the earpiece mic or headphones, and can only hear them if you're on speakerphone, or (b) the other person can't hear you if you're not using speakerphone?