Sadly you can't disable it - the feature is built into apps that support Chromecasts. You could turn off the Chromecast / unplug it then the feature won't be there.
That is unfortunate. When watching TV, I periodically get silly YouTube kid videos when the toddler hits that cast button. Happens to both my Chromecast and Roku. Thank you.
If you go into YouTube Kids settings there is an option to disable casting to big screens. It is only works for that app. But it solved my problem. The button is completely gone now!