Same problem here.. I resolved it turning off the "always listen " option on the Google app. It seems like this option takes the control of the microphone all the time.
I get this solution when another apps notify me "another apps is using the microphone " ... Ok google XD