Yes, the cell modem will remain on, but it uses next to no power when not actively trying to communicate.
That's an interesting experiment.That's what I would've thought as well, but I've experimented on this at work -- with the cell radio on but still connected to wi-fi, the battery drain would be significantly worse than if I turned on Airplane Mode and then turned wi-fi back on. I did this for a little while, but then realized that wi-fi calling on wi-fi alone would lead to less reliable sound while on a call.