Samsung once again caught artificially boosting benchmarks, receives internet wedgies

Samsung got caught artificially boosting their benchmark results again, this time for the new Galaxy Note 3. Ron Amadeo of Ars Technica:

After a good bit of sleuthing, we can confidently say that Samsung appears to be artificially boosting the US Note 3's benchmark scores with a special, high-power CPU mode that kicks in when the device runs a large number of popular benchmarking apps. Samsung did something similar with the international Galaxy S 4's GPU, but this is the first time we've seen the boost on a US device. We also found a way to disable this special CPU mode, so for the first time we can see just how much Samsung's benchmark optimizations affect benchmark scores.

On one hand, who cares? Synthetic benchmarks don't mean as much as how fast a phone turns on, or powers down, or launches a game, or crunches a video, or does any of a thousand other things real people need them to do every day. Things that are far more important, and far harder to cheat at. On the other hand, given that, why bother? Especially if you've been caught before. You know you'll be caught again, and you'll give people who already dislike or distrust you yet another reason to pile on the internet wedgies, and all for absolutely no gain.

It's nothing worth cheating over. It's nothing worth Apple's SVP of worldwide marketing, Phil Schiller, tweeting over. It's certainly nothing worth breaking reviewer or customer trust over, not for a few extra points on a graph only nerds read anyway.

It's silly.

Have something to say about this story? Share your comments below! Need help with something else? Submit your question!

Rene Ritchie

Editor-in-Chief of iMore, co-host of Iterate, Debug, Review, Vector, and MacBreak Weekly podcasts. Cook, grappler, photon wrangler. Follow him on Twitter and Google+.

More Posts

 

-
loading...
-
loading...
-
loading...
-
loading...

← Previously

The case of the missing iPhone 5s hardware accents

Next up →

Apple working on a fix for 'fraction-of-a-percent' experiencing iMessage problems

Reader comments

Samsung once again caught artificially boosting benchmarks, receives internet wedgies

23 Comments

It once again shows how unethical a company Samsung is. As we all know, they do the same with their sales statistics as well. This is very ironic since Koreans supposedly pride themselves on honour.

++ unfortunately every other company is basically just as unethical. We need good guys in this industry who put the consumer over the shareholders.

What a joke Samsung. Why don't you spend time copying apple where it matters and that is in the finer details. If you really want to be Android's Apple then you would spend more time and money making your software run better then messing with the processor to get a few more points on a benchmarking app that Apple clearly laps you in every year with their A series processors. Not to mention clocked at half the Ghz, half the cores, and 1/3 the amount of RAM. Innovate don't cheat.

But Samsung would need something worth copying first right? Last I checked the quick settings pull up drawer in iOS 7 was Google's idea. Nice try though...

Well Samsung was doing quick settings in Touchwiz before Android stock did. They had it in Eclair on the OG Galaxy S where Google finally added it in 4.2 on the Nexus 4

Well put....HTC also did it with the Sense on the Incredible 2 before Google baked it into Android. So here we have your Samsung example & my HTC example yet the douche that thought he was so smart has disappeared......shocking ;)

webOS, actually, is where that came from.. As was multi-tasking cards and Androids notification center. Google just copied it first.. so in that sense.. Google was the first to copy it?

"On one hand, who cares? Synthetic benchmarks don't mean as much as how fast a phone turns on, or powers down, or launches a game, or crunches a video, or does any of a thousands of other things real people need them to do every day."

Benchmarks are used to characterize the performance of system. Doing it with "full" applications (like a Handbrake transcode, measuring a suite of app launch times, etc) and such is a rather gargantuan task and will likely take well over the review period (though I don't think this is true). With benchmarks, the characterization of the system can be done in a much shorter time. They are supposed to be designed to replicate the tasks that a user does.

By characterizing the performance of a devices in a consistent way, it allows consumers to compare and contrast the devices they are looking at and enables them to make an informed decision. That the Smartphone Experts Network, the Verge, and other sites I don't visit have a policy not to publish them only decreases my confidence that the reviewers know what they are talking about, especially vis-a-vis device comparisons.

As a consumer, I put zero trust into anything a Verge review or its ilk says these days as they don't state or establish any facts to back up their platitudes. The benchmarks give truth and context to the statements like "this phone is fast!" I'd like a device to be weighed and its dimensions measured even, and as far as I can tell, that's only be done once, and it was for sensationalism purposes with the white iPhone 4 manufacturing tolerances.

That benchmarking is gamed isn't a new thing. That gadget reviewers think benchmarking is worthless is, what, symptomatic of the reviewer not really interested in technology and not taking the steps to understand the technology they review? Heck, reviewers should be all in on understanding performance. Specifications are touted incessantly yet when it comes time to understand performance, the only thing that is important, nobody actually does it or does it well.

Benchmarking characterizes performance. If the benchmark is gamed, do it in another way.

Benchmarks are important in todays market. It allows customers to differentiate among products. Cheating on benchmarks is like saying "Our phone takes 20mega pixel photos" when they only take 8 or 13. It sways consumer opinion. "This phone is the fastest, I could get this one" but in truth, it's no better than the competitors product.

Samsung cheats because it's their corporate nature.
It's just another part of their vast dirty tricks arsenal.

They cheat because the real world performance is hobbled by their custom UI. If it isn't now, it will be, whenever the phone receives updates. But cheating on bench marks that really don't matter to real people in the real world, who aren't propeller heads? Waste of time and resources.

LMAO.....how insightful!!!!! Someone please give this user a job writing a weekly op-ed on iMore............

I'm going to the mall and ask everyone I see how they feel about this. I would be willing to bet no one cares. Or even knows what benchmarks are. Carrying on over this is beyond silly. It's ludicrous.

The amazing thing is how Samsung gets a pass on this. If Apple did this, the Senate would actually pass the budget, get back to work so they could convene a hearing on how Apple is ruining America's schoolchildren ... but Samsung, no big deal. EVERYONE does this, right? right? Android, Oracle's borrowed OS, lowering ethical standards at twice the usual speed!

I think the funniest thing in all of this is that Samsung actually took the time to BUILD a cheat into the CPU before putting it on the market knowing that people were going to test it! That is pure genius and cheating innovation at its best!

"Win if you can, lose if you must, but always cheat!"