Why do benchmark scores vary so much?

[804hokie]

I've used the Quadrant Standard app for sometime to give me system info and benchmark it against other Android devices. However, it seems everytime I run a full system benchmark, the score seems to change from the previous test. For example, my rooted Bionic's best score on ICS was 2404. However, when upgrading to JB, my initial benchmark was 2477. But the best score I've seen on my Bionic with JB was 2560. I've seen it as low as 2156 on JB, too.

 

[meyerweb#CB]

It shouldn't be surprising that the numbers change when you change OSs. One would hope that newer versions of the OS have better optimizations than older ones. Part of JB, I believe, was what Google called Project Butter, which was all about performance enhancement.

Aside from that, these are multitasking OSs, so there are probably always other tasks going on in the background that might affect the result, giving slightly slower or faster results from one test to another. And the benchmarks themselves are far from perfect. At best, they give you a rough idea of how different devices compare. One shouldn't read too much into the results.