Benchmark cheating resurfaces with OnePlus and Meizu caught in the crosshairs

A few years ago a wave of cheating on benchmarks rolled through the smartphone community thanks to almost all manufacturers trying to obtain top marks on benchmarking tools. As the extent of the cheating was revealed and users became increasingly disillusioned with claims by manufacturers, the companies engaging in the cheating eventually figured out their actions were not worth the customer backlash they were creating. Unfortunately, some recent testing suggests this ugly trend may once again be rearing its head.

The latest round of apparent cheating comes to us via OnePlus. The OnePlus 3T was undergoing some testing by XDA Developers using, amongst other tools, Geekbench. The team conducting the tests noticed the cores in the processors in the OnePlus 3T devices being tested were not operating as traditionally observed. After some consultation with Primate Labs, creators of the Geekbench test, the parties verified that the OnePlus 3T cores were operating in a non-standard manner when benchmarking apps were detected as running.

When confronted with the results of the research, OnePlus at least had the backbone to admit that their code did in fact check for benchmark tests and would then adjust how the processors operated. OnePlus claims this was not done to cheat on benchmark tests, but instead was done to ensure users received the best experience and took advantage of available hardware when needed. This would include times when the smartphone may be used for gaming. OnePlus says they are planning to continue using this model of adjustments to their processor operations, but they would put an end to their system doing the same thing when benchmarking apps are detected. The XDA Developers note that the OnePlus solution creates an artificial limitation for users who have to rely on the operating system being updated for new, processor-intensive applications. The preferred method for triggering resources ramping up their operations is to have the system respond to increased demand rather than just an app’s name.

Testing revealed the Meizu Pro 6 is utilizing a similar method for turning on the big cores in their chips – looking for specific app names. Perhaps a little bit to their credit, Meizu at least provides a user-facing option to place the device in “performance mode” or “balanced mode.” The XDA Developers indicate they still think the performance mode specifically looks for benchmarking apps. Further, the performance mode does not appear to be a setting that can be used for normal day-to-day use, even by heavy gamers.

Hopefully these instances of manufacturers ratcheting up performance in response to benchmarking apps is not a trend that is returning to the market. Benchmark scores nearly became a useless tool for buyers during the last round of cheating and it would be unfortunate to return to that state.

source: XDA Developers


About the Author: Jeff Causey

Raised in North Carolina, Jeff Causey is a licensed CPA in North Carolina. Jeff's past Android devices include an HTC EVO, a Samsung Note II, and an LG G3, and a Motorola Moto X Pure Edition along with a Samsung Galaxy Tablet 10.1. He currently uses a Samsung Galaxy S8 and (very rarely) a Nexus 7 (2013). He is also using a Verizon-branded Motorola Moto Z Play Droid supplied by his job. Jeff used to have a pair of Google Glass and a Moto 360 Sport in his stable of gadgets. Unfortunately, his wife and kids have all drunk the Apple Kool-Aid and have i-devices. Life at home often includes demonstrations of the superiority of his Android based devices. In his free time, Jeff is active in his church, a local MINI Cooper car club, and his daughter's soccer club. Jeff is married, has three kids, and a golden retriever.