Page 1 of 2 12 LastLast
Results 1 to 25 of 33
Like Tree16Likes
  1. Thread Author  Thread Author    #1  
    Ry
    Ry is online now
    Ry's Avatar

    Posts
    7,195 Posts
    Global Posts
    7,202 Global Posts

    Default Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    a.k.a. RyZR from HoFo
    Motorola Moto X (164.55.2.ghost_verizon.Verizon.en.US, KitKat 4.4.2)
    Motorola Moto G (173.44.20.falcon_cdma.Verizon.en.US, KitKat 4.4.2)
    Motorola DROID Bionic (98.72.22.XT865.Verizon.en.US, Jelly Bean 4.1.2)
    Google Nexus 7 2012 (KOT49H, KitKat 4.4.2)
    What other devices have I had? Here's my phone timeline.
    Support your favorite Android app and game developers. Pay for apps! And don't block ads!

    Community Rules & Guidelines - Mobile Nations Forums
  2. #2  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Yawn
    crester likes this.
  3. #3  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    wow...this is on almost every site I visit and the news is just getting here. Anyways Samsung needs to stop with this crap it's very misleading. How about letting all apps and games run at those clocks and not just benchmarks.
  4. #4  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    That's LAME Samsung, LAME. Why would you act to make owning your product an embarassment?
  5. #5  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    This is such a non issue. As the Anadtech review says, most Android phones do this. WHO CARES?!
    msndrstood and crester like this.
  6. #6  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Better question is why would any benchmark have cores idling or at 1/8th full speed? So this little code seems to not allow the cores to throttle down once the app is started. I suspect that they just don't need to speed up during a test and never turn down like it would otherwise. Not sure if that is whats giving the 20% boost, but the lack of throttling must help during testing.
  7. #7  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    So when it's in power saving mode it will let 3 of the cores go idle, but when there's an intense workload it will turn on all 4 cores?

    Pardon me, but that isn't a bug, it's a feature. You save battery when you don't need it, and when you do need it, all four cores of the CPU will actually be engaged.

    Seems pretty normal Linux kernel behavior to me. Which is what Android is built on top of.

    If the reviewer's going to complain about anything, they should complain about using synthetic benchmarks in the first place.
    msndrstood and Cary Quinn like this.
  8. #8  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    PC Gamers have been doing this a long time. Ask any serious gamer if the leave Intel Speedstep or AMD's Cool n' Quite enabled on there gaming rig.
  9. #9  
    Tsepz_GP's Avatar

    Posts
    99 Posts
    Global Posts
    125 Global Posts

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    AnandTech's Review:


    "CPU performance is honestly excellent. The Galaxy Note 3 is more or less the fastest Android smartphone we've tested up to this point. In the situations where we can do cross platform (OS/browser) comparisons, it isn't quite as fast as the iPhone 5s but in some cases it comes close. I should mention that the Note 3 (like many other Android devices - SGS4, HTC One) detects certain benchmarks and ensures CPU frequencies are running at max while running them, rather than relying on the benchmark workload to organically drive DVFS to those frequencies. Max supported CPU frequency is never exceeded in this process, the platform simply primes itself for running those tests as soon as they're detected."

    It's also interesting to note that the Galaxy Note 3 appears to outperform all other Snapdragon 800 smartphones we've tested thus far. There's a couple of potential explanations here. First, the Galaxy Note 3 is using newer drivers than any of the other S800 platforms we've tested:

    Note 3: 04.03.00.125.077
    Padfone: 04.02.02.050.116
    G2: 4.02.02.050.141"


    Move along folks, nothing to see here.
    Samsung GALAXY S 4 32GB Exynos Octa Black Mist
    Samsung GALAXY S II 16GB Ceramic White
    Apple iPad4 WiFi+3G/LTE 16GB
    Thanked by:
    Reggie 
  10. #10  

    Default

    According to Slashgear HTC and LG have done the same with the G2 and the ONE. So samsung is not the only one they are just getting all the negativity while everyone else get a free pass.

    Posted via Android Central App
  11. #11  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Blackintuit View Post
    So when it's in power saving mode it will let 3 of the cores go idle, but when there's an intense workload it will turn on all 4 cores?

    Pardon me, but that isn't a bug, it's a feature. You save battery when you don't need it, and when you do need it, all four cores of the CPU will actually be engaged.

    Seems pretty normal Linux kernel behavior to me. Which is what Android is built on top of.

    If the reviewer's going to complain about anything, they should complain about using synthetic benchmarks in the first place.
    not even the same thing as a PC. A PC doesn't raise frequencies high on just benchmarks. It does it with games also which the Note 3 doesn't. All it does is raises the frequency during benchmarks and nothing more.
  12. #12  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by apollooff320 View Post
    not even the same thing as a PC. A PC doesn't raise frequencies high on just benchmarks. It does it with games also which the Note 3 doesn't. All it does is raises the frequency during benchmarks and nothing more.
    How do you know what it does in the games? Have you tested or do you make such assumption because of some random named non-game program?

    Posted via Note3
    Cary Quinn likes this.
  13. #13  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    I know on laptops, you can have power management options to have the device run the cpu in a lower frequency range when on battery than when plugged in.

    Android devices have had CPU Tuner as an option since Android 2.1 (on rooted devices). As was pointed out above, the OS has built in power management functions and most, if not all, devices run the CPU in more of a battery saving state by default, but has the option to boost the CPU to full speed if needed. Most of the time it should not be needed, even on games, to get good performance out of the device.

    The best takeaway I get from the ARS article (which I am thankful for their testing, even though their conclusions seem biased against Samsung for giving the benchmark tool what it is asking for); is that even with the benchmark being treated as a generic program the Note 3 compares favorably against other devices using the same CPU.

    I like the idea of having the most kickass CPU on the market right now in my new phone; but I'd much rather also have the idea that the battery life of that phone is being maintained so I can run it for days at a time without having to plug it in everywhere I go.
  14. #14  
    Sooks's Avatar

    Posts
    526 Posts
    Global Posts
    5,131 Global Posts

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    I dont understand what people aren't understanding? When I run a benchmark on my PC all cores become unparked the speed hits turbo boost and max out, my graphics card spins up to it highest clock speed. It is doing exactly what it is supposed to do during a benchmark test.
    Thanked by:
  15. #15  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    I tend to agree with you however I kind of understand what some other are concerned about (the inability to ramp all four cores for other situations). Still though its a benchmark and in the pc world people are using LN2 just to reach insane speeds. What people need to realize is a benchmark mark is stressing the system to see its potential not for its everyday usability.
    Sent from my SGH-T889 using AC Forums mobile app
  16. #16  
    Sooks's Avatar

    Posts
    526 Posts
    Global Posts
    5,131 Global Posts

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Exactly, and there is no way a 2.3 GHz Quad-Core with 3 GB of RAM wont keep up on day to day task.
  17. #17  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by apollooff320 View Post
    not even the same thing as a PC. A PC doesn't raise frequencies high on just benchmarks. It does it with games also which the Note 3 doesn't. All it does is raises the frequency during benchmarks and nothing more.
    It doesn't raise frequencies at all, it just makes them run at full speed, there is no over clocking, the chips are still running at 2.3 ghz, it just turns off all the power saving features. It doesn't do this with anything else because a) nothing needs the full power of the chip to run properly and b) running like this for any length if time will eat your battery
  18. #18  
    Kevin O'Quinn's Avatar
    AC Team Emeritus

    Posts
    8,729 Posts
    Global Posts
    8,906 Global Posts
    ROM
    Stock

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Tsepz_GP View Post
    AnandTech's Review:


    "CPU performance is honestly excellent. The Galaxy Note 3 is more or less the fastest Android smartphone we've tested up to this point. In the situations where we can do cross platform (OS/browser) comparisons, it isn't quite as fast as the iPhone 5s but in some cases it comes close. I should mention that the Note 3 (like many other Android devices - SGS4, HTC One) detects certain benchmarks and ensures CPU frequencies are running at max while running them, rather than relying on the benchmark workload to organically drive DVFS to those frequencies. Max supported CPU frequency is never exceeded in this process, the platform simply primes itself for running those tests as soon as they're detected."

    It's also interesting to note that the Galaxy Note 3 appears to outperform all other Snapdragon 800 smartphones we've tested thus far. There's a couple of potential explanations here. First, the Galaxy Note 3 is using newer drivers than any of the other S800 platforms we've tested:

    Note 3: 04.03.00.125.077
    Padfone: 04.02.02.050.116
    G2: 4.02.02.050.141"


    Move along folks, nothing to see here.
    There's absolutely plenty to see here. The fact that many manufacturers are doing this doesn't excuse it, either. It wasn't ok with the PC market, and it shouldn't be ok with the mobile market.
    Kevin F.I.M.T.K. O'Quinn Esq.
    Johnly and Ry like this.
  19. #19  
    Sooks's Avatar

    Posts
    526 Posts
    Global Posts
    5,131 Global Posts

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Kevin O'Quinn View Post
    There's absolutely plenty to see here. The fact that many manufacturers are doing this doesn't excuse it, either. It wasn't ok with the PC market, and it shouldn't be ok with the mobile market.
    Why are we comparing apples to androids ?

    The iphone pushes sub 720p .... the note 3 is pushing 2.5 times the pixels... no **** it is going to take a split second faster to load
  20. #20  
    Kevin O'Quinn's Avatar
    AC Team Emeritus

    Posts
    8,729 Posts
    Global Posts
    8,906 Global Posts
    ROM
    Stock

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Sooks View Post
    Why are we comparing apples to androids ?

    The iphone pushes sub 720p .... the note 3 is pushing 2.5 times the pixels... no **** it is going to take a split second faster to load
    I wasn't aware that I compared anything to Apple. I made a statement that companies "cheating" on benchmarks should be unacceptable.

    It's a little bit ironic that nVidia isn't cheating on any of them, at least according to Anandtech. Maybe they actually learned something from going through it back in the PC days.
    Kevin F.I.M.T.K. O'Quinn Esq.
  21. #21  
    Sooks's Avatar

    Posts
    526 Posts
    Global Posts
    5,131 Global Posts

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Kevin O'Quinn View Post
    I wasn't aware that I compared anything to Apple. I made a statement that companies "cheating" on benchmarks should be unacceptable.

    It's a little bit ironic that nVidia isn't cheating on any of them, at least according to Anandtech. Maybe they actually learned something from going through it back in the PC days.
    \

    Sorry not you specifically but from AnandTech.

    Also, how is this cheating?
  22. #22  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    I don't understand how ramping up the processors to 100% is cheating, sure its not indicative of everyday use but it showing the potential SoC. Its not like they are circumventing part of the benchmark just running their chip full potential. I guess its all in how you view the subject but a little transparency would surely help.

    Sent from my SGH-T889 using AC Forums mobile app
  23. #23  
    Kevin O'Quinn's Avatar
    AC Team Emeritus

    Posts
    8,729 Posts
    Global Posts
    8,906 Global Posts
    ROM
    Stock

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Sooks View Post
    \

    Sorry not you specifically but from AnandTech.

    Also, how is this cheating?
    Quote Originally Posted by jrham View Post
    I don't understand how ramping up the processors to 100% is cheating, sure its not indicative of everyday use but it showing the potential SoC. Its not like they are circumventing part of the benchmark just running their chip full potential. I guess its all in how you view the subject but a little transparency would surely help.

    Sent from my SGH-T889 using AC Forums mobile app
    It's not overclocking, like was discovered on the S4, which is technically worse. I'll try to explain it. During normal operation the CPU is constantly changing speed based on demand. That's great for power management. It means that the CPU can idle most of the time, and ramp up speed when you're doing stuff. If you were to look at a graph of CPU speed doing anything (and I mean anything, since nothing will ever peg the CPU to max speed continuously) it'll have peaks and valleys. Kind of like a heart monitor. It's very dynamic and is almost constantly changing, especially with the current crop of quad-core CPU's. Each core will also be able to operate dynamically, which helps with power draw and thermal management. That last point is something I will touch on further down.

    The goal of a CPU is to get work done as fast as possible and then get back down to the idle state, where it's drawing the least amount of power. "Race to idle" is a term you may have heard before, and that's what it's referencing. You open an app and the CPU speed spikes, then goes back down. It did what it needed to do and went back to an idle state (or the slowest it could go while still giving good performance for what you're doing). So we've established that CPU speed is determined by the demands that you're putting on your device at that specific time. There's one other factor, though, and that's thermal management.

    As you know, CPU and GPU's generate a decent amount of heat doing there job, especially when being pushed very hard. Games and benchmarks are apps that would stress both components. Manufacturer's have targets for thermals, and a max temp and time that temp can be maintained. Once that point is reached it will "throttle" the CPU/GPU, which basically means it will slow them down. That's for longevity and is something that's very common across different sectors. If you did any reading on the S4 Pro in the Nexus 4 specifically, you'll know that it was thermally limited when running benchmarks. LG/Google let the chip do all the management and didn't smooth performance out. Other manufacturer's are more conservative, and will establish limits that will give more consistent performance. Odds are that they know that most users would rather have a consistent experience over spurts of faster performance.

    Based on what we know is going on then, OEM's have introduced a level of thermal and power management not available to any other app. While during normal operation a CPU/GPU will race to idle, when running a benchmark the speed is set to max the entire time. This prevents the CPU from powering down, which helps performance because it then doesn't have to speed back up. Over time, those milliseconds add up to lower benchmark scores. But what about the thermals? You can't just keep the speed at max and also keep thermals in check, right? They thought of that too, and have also changed the thermal thresholds (most likely) for this particular level of performance.

    So no, they aren't overclocking. What they're doing, and this isn't just Samsung, is introducing a performance tier that's only available to the whitelisted apps. That brand new game that you just picked up that has amazing graphics will never see the type of performance that the benchmarks are seeing, because they aren't on the whitelist. The CPU/GPU will operate dynamically just like they would for another other app, and would only see max speed across all four cores on rare occasion.


    Hopefully that all makes sense.
    Kevin F.I.M.T.K. O'Quinn Esq.
    Thanked by 3:
    Eclipse2K and Golfdriver97 like this.
  24. #24  

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Kevin O'Quinn View Post
    So no, they aren't overclocking. What they're doing, and this isn't just Samsung, is introducing a performance tier that's only available to the whitelisted apps. That brand new game that you just picked up that has amazing graphics will never see the type of performance that the benchmarks are seeing, because they aren't on the whitelist. The CPU/GPU will operate dynamically just like they would for another other app, and would only see max speed across all four cores on rare occasion.


    Hopefully that all makes sense.
    The question that should be asked then, is are we sure those other apps are not on the "whitelist", and if not, is there a way to add them to the list? Or is there simply some way to tweak the OS to open up more than "normal" levels of performance?
  25. #25  
    Sooks's Avatar

    Posts
    526 Posts
    Global Posts
    5,131 Global Posts

    Default Re: Note 3’s benchmarking “adjustments” inflate scores by up to 20%

    Quote Originally Posted by Kevin O'Quinn View Post
    It's not overclocking, like was discovered on the S4, which is technically worse. I'll try to explain it. During normal operation the CPU is constantly changing speed based on demand. That's great for power management. It means that the CPU can idle most of the time, and ramp up speed when you're doing stuff. If you were to look at a graph of CPU speed doing anything (and I mean anything, since nothing will ever peg the CPU to max speed continuously) it'll have peaks and valleys. Kind of like a heart monitor. It's very dynamic and is almost constantly changing, especially with the current crop of quad-core CPU's. Each core will also be able to operate dynamically, which helps with power draw and thermal management. That last point is something I will touch on further down.

    The goal of a CPU is to get work done as fast as possible and then get back down to the idle state, where it's drawing the least amount of power. "Race to idle" is a term you may have heard before, and that's what it's referencing. You open an app and the CPU speed spikes, then goes back down. It did what it needed to do and went back to an idle state (or the slowest it could go while still giving good performance for what you're doing). So we've established that CPU speed is determined by the demands that you're putting on your device at that specific time. There's one other factor, though, and that's thermal management.

    As you know, CPU and GPU's generate a decent amount of heat doing there job, especially when being pushed very hard. Games and benchmarks are apps that would stress both components. Manufacturer's have targets for thermals, and a max temp and time that temp can be maintained. Once that point is reached it will "throttle" the CPU/GPU, which basically means it will slow them down. That's for longevity and is something that's very common across different sectors. If you did any reading on the S4 Pro in the Nexus 4 specifically, you'll know that it was thermally limited when running benchmarks. LG/Google let the chip do all the management and didn't smooth performance out. Other manufacturer's are more conservative, and will establish limits that will give more consistent performance. Odds are that they know that most users would rather have a consistent experience over spurts of faster performance.

    Based on what we know is going on then, OEM's have introduced a level of thermal and power management not available to any other app. While during normal operation a CPU/GPU will race to idle, when running a benchmark the speed is set to max the entire time. This prevents the CPU from powering down, which helps performance because it then doesn't have to speed back up. Over time, those milliseconds add up to lower benchmark scores. But what about the thermals? You can't just keep the speed at max and also keep thermals in check, right? They thought of that too, and have also changed the thermal thresholds (most likely) for this particular level of performance.

    So no, they aren't overclocking. What they're doing, and this isn't just Samsung, is introducing a performance tier that's only available to the whitelisted apps. That brand new game that you just picked up that has amazing graphics will never see the type of performance that the benchmarks are seeing, because they aren't on the whitelist. The CPU/GPU will operate dynamically just like they would for another other app, and would only see max speed across all four cores on rare occasion.


    Hopefully that all makes sense.
    I'm a Network Engineer / Systems Administrator and it makes perfect sense. However I still don't see the point. The entire purpose of a benchmark is to run everything at max to see what the highest potential is (never expecting to run constantly at that speed/temp therefore never expecting to run at that potential) My system has SLI 780s which you know very well, when I 3D Mark them, they run hotter and faster then I ever intend to run them while working / gaming. However it is still fun to see what the max is. In regards to computers even the latest i7s park half the cores during most events, you have to go into the registry and force then to never park. I feel like this is standard practice for wanting to see how things run at absolute max and in my book is a fair benchmark.
Page 1 of 2 12 LastLast

Similar Threads

  1. Changing battery's better to shutdown the S4?
    By Nuno Mota in forum Samsung Galaxy S4
    Replies: 5
    Last Post: 10-02-2013, 12:15 PM

Posting Permissions

B