Sorry, but I'm a bit stumped by this. Go to Google Calendar on the desktop browser, click the gear icon at the upper right, then Settings. What is "Automatically add invitations to my calendar" set to?
As a workaround, have you considered sharing each other's entire calendars with each other? You can then toggle your husband's calendar on and off easily from within your Calendar app. This is what my wife and I do -- if I have to check for conflicts, I simply toggle her calendar on briefly, review the events she's scheduled, then toggle it back off again.