I think Jao36 might be right, its a bunch of lint. I had the same problem with my jack not clicking in and falling out all the time. It was getting very annoying. So because I keep my phone in the front pocket that seemed like a likely cause. So how I fixed it was a took a small sewing pin, put it in a vice, and gently hit the tip with a hammer. This bent the sharp tip into a very small hook. Then I worked this in and out of the hole and yes, there was a ton on lint inside that I couldn't even see. After that it clicked like new.