A lot more people than you think try to do research on their tech purchases and the benchmark results filter down to them in this way. People will google around, read Consumer Report, etc. and hear "this phone was the fastest" and this will inform the buying decision. This is shady, shady stuff. Not cool.
Samsung cheats at benchmarks
- Thread starter snookasnoo
- Start date
vferrari
Well-known member
Thank goodness for this post because with all the sports analogies, I thought this thread had jumped the shark.
Anyhoo, allow me to take exception to the proclamation: "A lot more people than you think..." Someone needs to publish the market research that says a significant percentage of phone consumers make their decisions based on published benchmark results. Because regardless of whether you buy what Samsung is selling (I am referring to their "cover story" on what is really going on with the GPU clock) or think they are reprehensible crooks, if the benchmark results are not really appreciably moving the meter on sales, then this is all just a great geek debate.
Last edited:
JHBThree
Well-known member
This is the situation where it makes a difference:
A consumer reads reviews on a site like Engadget for the S4 and One. In the review, they notice the S4 has higher scores in most of the benchmarks. Because of that, they assume the S4 is faster than the One, and therefore a better phone. They end up purchasing the S4 instead of the One, with that speed being a reason.
Will every consumer do that? No. But those that go to sites that publish these benchmarks will.
Sent from my SGH-M919 using Tapatalk 2
vferrari
Well-known member
If they can't tell they've been scammed base on hands-on experience with the phone, does it matter?
In other words, if the only way you can tell whether one device edges another in performance is through a benchmark and not through real hands on use, does it matter (for a consumer device)?
Last edited:
monsieurms
Well-known member
I don't like black and white arguments. Of course, benchmarks matter if they are used and interpreted correctly. Conversely, all those "real world" anecdotes that are so popular here are often (a) biased (b) from uncontrolled tests and unmeasured; (c) from, sometimes, people who don't understand the device or the software; or (d) from configurations that don't reflect what the phone is doing (or not doing), only what other things in the configuration are doing to affect it. Benchmarks avoid subjectivity and bias and they control for results by eliminating other possible causes. Conversely, from reading this forum at times, for example--I'm remembering the headline of one thread--the S4 must be a total failure, an "overrated lag machine." That is a completely deceptive comment that misleads in terms of depicting how fast this phone really is. Frankly, the benchmark tests, even as exaggerated, were a lot more accurate in conveying the real truth of how fast this phone is than comments like that. And it is lightning fast. If I didn't come to this forum, I wouldn't have known I should be complaining about it.
Which brings up the next, related point. While I've expressed annoyance at ANY manipulation just on principle, the bottom lines here are that
(a) this seems to be "par for the course" marketing; and
(b) the picture painted by the benchmark results is pretty much consistent with the phone's performance in the real world.
In other words, the marketing message conveys an essentially correct message even if obscure details are exaggerated. The phone is simply blazing fast. It makes my last HTC look like a 1999 model. It makes my Asus tablet look like it is 10 years out of date. It sometimes outperforms my XPS desktop. The benchmark marketing as such basically conveys, if you want to put it in Colbert fashion, a truthiness. They may've gone overboard. They may've played the marketing game. But the phone is blazing fast.
That's why the Samsung "fanboys" aren't so pissed off in this thread. We don't feel particularly ripped off. The phone performs as we expected a top of the line, super fast phone to perform in terms of speed gains. We don't really feel particularly misled in essential respects. We didn't, in other words, buy a Porsche only to find out that it is a Smart Car. Even if the specs may not be completely accurate, it's still a Porsche and has so much power to burn that it is hard not to be impressed with its acceleration.
Last edited:
That's quite interesting... I'm now wondering if other phone OEMs do so as well. Then again, I've always judged a phone by its reviews (both media and customer ratings), and also just skimmed through the specs to get an image of the device's power (but sometimes, specs aren't always accurate to how good a device performs).
So based on this thread you still believe marketing material (which is what that video is) is 100% true? Here's just one blog post I found regaring professional photography and food:
How To Take Great Photos Of Food - Business Insider
I also know a professional photographer who's done pictures of food and most of what he's shot, or anyone he knows has shot, is not edible.
When was the last time you went to McDonalds and got a quarter pounder with cheese that looks like this?
http://www.mcdonalds.com/content/dam/McDonalds/item/mcdonalds-Quarter-Pounder-with-Cheese.png
The ones I get look more like this:
http://4.bp.blogspot.com/-dUuAPBdx9...arter+Pounder+with+Cheese+Ketchup+Mustard.JPG
The marketing picture is not an accurate representation of what you actually get when you order.
Fast is relative. If you like your phone, and it runs your apps, then it's fast enough. But this isn't about what speed the S4 is actually running at. It's about Samsung gaming the system - cheating - in an effort to sell more phones. This could seriously be a consumer protection class action.
I understand the point you're making. Practically, and legally, you're wrong. The proper comparison would be to say, look at this beautiful picture of a Quarter Pounder - with cheese - and then you go buy one - and it doesn't have cheese, it can't have cheese and it never will have cheese. It's perfectly LEGAL to make your widgets look good to sell them. It is NOT LEGAL to lie about what the widget IS in order to sell them.
Here's the best example I could find: Western Digital Settles Capacity Dispute - Breaking - Technology - smh.com.au
While there wasn't a factual finding by a judge or jury - they settled the case - and now, as we all know, a 1tb hdd does not have 1tb available storage. But now there's a disclaimer on the box.
In this case, Samsung made very specific modifications to their product in order to achieve a level of performance that is not available - out of the box - to consumers.
The whole point of this and the point you are missing is that the Food industry is out in the open and admit they do this.
That was what they stated in the video. In McDonald's case, they do use the exact same food though and they give you the whole process about how they make it look better.
Nothing hidden, the question was asked. They even show you that they even photoshopped part of the sloppy parts of the food and touch up the coloring.
I remember as a kid (over 30 years ago) the restaurant industry saying the same thing back then. We do make our food look better in pictures.
Here is the meat of the issue(pun intended).
Samsung did this in a hidden manner and still denies what they did. They got caught and they are still lying about this.
If Samsung from the very beginning came out and stated. In certain situations to improve performance our gpus will increase clock speed to 533mhz versus the soft cap of 480 mhz, then none of this would have mattered as much because they admitted from the beginning.
I'll let the rest of your slightly insulting post be someone else's problem to deal with. I just want you to tell us all how to "interpret benchmarks correctly."
slackerjack
Well-known member
No, I'm not going to agree with you on the target audience at all. Nvidia is clearly targeting the same console audience with the price point they've chosen. It's not the handheld market that's for sure - 3ds's can be had for around $150, Sony's Vita doesn't even come into the equation with it's abysmal adoption - putting it up squarely against the Wii U (at the same price point), which despite being the slow starter - is a LOT more console than the SHIELD, and of course the current consoles (at the same price point as the) - then against the next generation of consoles at $100 more. So who else is there? The cellphone/mobile gaming crowd? That's pretty well covered up by actual cell phones, and even the outliers are the iPod Touch and Galaxy Player models, not to mention the barrage of sub $400 tablets that offer bigger screens and more functionality.
Look, I'm not saying SHIELD is a terrible idea, but the timing couldn't be worse - we're <6 months away from the holiday buying season - it's actually already started, everyone interested in any sort of technology for games is fettering away their nickles and dimes for the next generation of consoles, and if they understand that they don't have the budget - fettering away those same nickles and dimes for the current generation of consoles. People interested in phones and tablets are buying those, so whom does this leave the SHIELD to market too? Little Jimmy, who really likes his mom's Galaxy S4/HTC One but wants a clunky controller and smaller (worse) screen to play all of his games on? Even if that we're true, and it'd have to be a very specific case, how many little Jimmy's are there out there versus little Tommy's who want the Xbox One or Ps4 - then put yourself in the parents shoes - "big all inclusive console that gets me games for my kids (or me), and all of this robust media center functionality? Or save $100 dollars and buy them a hand held that has fewer big name games and is twice as much as the 3ds" - talk about needing a hellacious marketing strategy...and one helluva "whiz bang" feature that no other device can offer.
Again, the Shield is a fine device, it's just limited in terms of what it's going to do for you versus what the consoles can do for you - and the transparency of what those things are. How on earth Nvidia plans to sell enough of those to make it turn any kind of profit is just beyond me, especially given the landscape of the technology business (which is quickly approaching landslide conditions....and not in the good way).
This sort of thing isn't terribly unusual. I recall how Intel modified one of its runtimes that is implemented by most PC software. Intel purposefully designed it to sneakily give terrible performance on AMD rigs. They denied it for ages before they finally settled. GPU companies design their hardware specifically to dominate benchmarks, even if that kind of performance doesn't carry on to regular use.
I'm not saying I like it, but this is hardly precedent.
Posted via Android Central App
I'm not saying I like it, but this is hardly precedent.
Posted via Android Central App
monsieurms
Well-known member
No problem! Interpreting benchmarks correctly means simply understanding their context, both pros and cons, advantages and limitations, which is clear enough, I hope, from the post. But if not, that's how you interpret benchmarks. It doesn't mean they are useless. It doesn't mean they are perfect. It also doesn't mean anecdotes here are the be-all, end-all, last word, either when they and benchmarks collide. They come with their own context and limitations as well. Oddly, only benchmarks get criticized.
It's the entire universe of data points that makes the most sense. But the post I made, responding to someone indicating everyone agrees benchmarks are useless, was in rebuttal of that comment. They aren't useless and in particular they have the advantage of being objective tests in controlled circumstances with neither bias nor subjectivity. They are an important piece in the analytical framework. That's why they exist.In any event, as I said in an earlier post, every single pro review I read included real world testing to go with benchmarks. They all came to same conclusions. Not one person ever said "Wow. According to the benchmarks, this is a really fast phone. But the rest of the time, it's barely useable." The benchmarks here and the actual performance seem pretty much in sync.
Last edited:
OperationA7X
Well-known member
Didn't they announce over at XDA that HTC is doing it too? Something about htclib-opt2 or something like that, that also white lists applications.
I'll find the source when I can.
EDIT: Found it. http://forum.xda-developers.com/showpost.php?p=36033515&postcount=744
Sent from my HTC Vivid BoRKeD using Android Central Forums
I'll find the source when I can.
EDIT: Found it. http://forum.xda-developers.com/showpost.php?p=36033515&postcount=744
Sent from my HTC Vivid BoRKeD using Android Central Forums
Last edited:
No problem! Interpreting benchmarks correctly means simply understanding their context, both pros and cons, advantages and limitations, which is clear enough, I hope, from the post. But if not, that's how you interpret benchmarks. It doesn't mean they are useless. It doesn't mean they are perfect. It also doesn't mean anecdotes here are the be-all, end-all, last word, either when they and benchmarks collide. They come with their own context and limitations as well. Oddly, only benchmarks get criticized.
In any event, as I said in an earlier post, every single pro review I read included real world testing to go with benchmarks. They all came to same conclusions. Not one person ever said "Wow. According to the benchmarks, this is a really fast phone. But the rest of the time, it's barely useable." The benchmarks here and the actual performance seem pretty much in sync.
I'll say they are 100% useless when they are run inside a virtual machine. On Android, they are. Compiled bytecode has no direct access to the hardware layer, it all goes through Dalvik.
Any interpretation is only of how well the DVM executes the bytecode, since we're not touching the hardware layer. This is something people tend to ignore, but they certainly shouldn't.
All you can benchmark is how well the DVM runs. This is why identical hardware (Optimus G/N4; SGS4/GPe S4; etc.) all return such wildly different results.
I've never ran a benchmark on my GS4. I do know its faster HTC One I had and that's all that really matters to me.
Posted via Android Central App on my Samsung GS4
Posted via Android Central App on my Samsung GS4
Look at the real picture. The whole burger doesn't have cheese on it. A good quarter of it is just a hamburger. This benchmark is showing the quality of a particular feature, not whether or not the feature is there. In your cheese example, the S4 would be able to run these types of programs in the benchmarks but not in practice. What actually happened is the benchmarks shows the feature running better in the benchmark than in practice, just like a burger in the picture is constructed out of the same materials in both pictures, but the quality and arrangement of them is much better in the advertisement than what you actually get.
The thing with the WD case is that WD was the one making the measurements. They claimed 80GB even thought it was only 74GB. Samsung isn't making the measurements, some other company is making the measurements. They aren't accurately measuring what the device can do. So Samsung is lying to the people that write the benchmarks and the people who write the benchmarks are lying to us.
They look like the same products to me.. One just is presented better.. I'm hungry now so I'm going to go grab a hamburger
Sent from my HTC One using Tapatalk 4 Beta
slackerjack
Well-known member
Similar threads
Trending Posts
-
[Chatter] "Where everybody knows your name ..."
- Started by zero neck
- Replies: 53K
-
-
-
-
Facebook
Twitter
Instagram