Thought I'd update this with a possible solution. Might help someone else with this issue.
As I said earlier this would happen mostly when I was taking my phone out of the holster. The holster has 2 pretty strong magnets and I got to wondering if that was having an effect. I took the holster...