I have the exact same problem. Same scenario even. Long time iphone user, just switched to android. I am VERY happy with the S5 aside from this slight issue.
Mine is a pretty noticeable ticking sound when I move the phone around, and sometimes when I tap. I don't have to flail the phone or tap...