So everyone posted that turning off "Ok Google" detection and "Google Now" cards solved this issue for them. But for me the solution was to turn off the stupid service of HD calling by ATT.
Go To
Settings > Mobile Data > Uncheck Enhnced LTE Service (HD voice calling)
Hope this helps