After some playing around with it, it's something I can get used to. For photos, OIS is fantastic. For video, it's always trying to keep the lens on point, so to speak. When you start to pan, it'll try to keep the lens pointed where it was until you physically move past its mechanical limits. Then it'll pan around ok, but you will see a noticeable delay from when you start panning until the image moves to match.
Once it's moving in a pan, it seems to be mostly ok with not much jello if your movements are smooth. If you're constantly jerking the phone about with abrupt changes, then you'll see more jello. If you really want to see something trippy, prop the phone up on something that is vibrating a lot so that it's also vibrating the phone. The view will look like the world is made of liquid swirling about.