Hmm.... for those who experienced the same audio issues on the Note 3, it looks like they've got a light at the end of their tunnel:
Sprint's Samsung Galaxy Note 3 gets update for audio fixes, keyboard improvements | Android Central
I'm sure a fix for the Verizon version won't be far behind...