I have this same issue. Everywhere the solutions people are proposing fail. The problem isn't something obvious like updating or buying new headphones, its a problem with the google app. No idea why the assistant does this, why not have an option where turning it off literally turns it off...