It's not really for casting local files. When you're trying to cast a local file you are first streaming it "out", then the cast is streaming it back "in".. If that makes sense..It's bound to have a lag.
It's not the same as Apple tv where you are streaming directly from phone to Apple TV. Chromecast is just pulling down online content that you tell it to.
"easily enjoy your favorite online entertainment on your HDTV"
-https://play.google.com/store/devices/details?id=chromecast