How to control IoT devices on a Guest Network?

[Mr MnmlEngr]

As I'm sure a lot of IoT device owners have read, it seems to be good advice to compartmentalize IoT devices on their own Wi-Fi network at home separate from your primary network with things like network drives, computers, phone, etc. And since routers that support more than two SSIDs are hard to find (or prohibitively expensive), I've read that the best practical way of accomplishing this is by putting them on the Guest network that many routers support.
I'm finally accruing enough IoT devices that I want to switch them to my guest network. But before I do, I'm having trouble visualizing the use concept...
Outside of voice-controlled devices (Alexa, Google Home speakers, etc), IoT devices typically are controlled by a smartphone. For example, Philips Hue lights. (Yes, I know that they can be turned on an off with Google Assistant, but a lot of their deeper features like scenes, and lab formulas, are only available in their app).

So if I need to control the devices with my phone, which needs to be on the same network as the IoT devices, and my phone has access to things like Google Drive, and files that are synced with my computer...where is the added security? Am I missing something?

I'm just wondering how other IoT owners that have their devices isolated on their own network have theirs set up.

Thanks in advance for the help!