Unfortunately, that seems to be a limitation of the way Chromecast (or GoogleCast) audio currently works; you have to stop casting before casting to a different speaker or multiroom group. I hope that Google can modify that to work more like Sonos where you can transition between speakers and...