Moto X 2013 here, my 2014 is coming in tomorrow.
I enabled encryption on my X2013 a few weeks ago, and I'm running ART on the last KitKat (4.4.4) so everything I'm doing is as close to Lollipop as possible without having Lollipop.
ART makes the executables larger, which plays directly against good performance in an encrypted environment where you have to read those executables at app startup. Once an app is started up, it runs faster.
So I'm seeing exactly what I would expect. When I start a relatively large app, there can be a delay of a half-second or so, then it runs nice and quick. If an app is already in memory and I'm task-switching to it, it's instant. If the app has been paged out of RAM, I get the delay again. Running many very large apps certainly means they page out more often, and I see lag. Running fewer large apps means they tend to stay in RAM, and I see buttery smoothness.