A few years ago a wave of cheating on benchmarks rolled through the smartphone community thanks to almost all manufacturers trying to obtain top marks on benchmarking tools. As the extent of the cheating was revealed and users became increasingly disillusioned with claims by manufacturers, the companies engaging in the cheating eventually figured out their actions were not worth the customer backlash they were creating. Unfortunately, some recent testing suggests this ugly trend may once again be rearing its head.
The latest round of apparent cheating comes to us via OnePlus. The OnePlus 3T was undergoing some testing by XDA Developers using, amongst other tools, Geekbench. The team conducting the tests noticed the cores in the processors in the OnePlus 3T devices being tested were not operating as traditionally observed. After some consultation with Primate Labs, creators of the Geekbench test, the parties verified that the OnePlus 3T cores were operating in a non-standard manner when benchmarking apps were detected as running.
When confronted with the results of the research, OnePlus at least had the backbone to admit that their code did in fact check for benchmark tests and would then adjust how the processors operated. OnePlus claims this was not done to cheat on benchmark tests, but instead was done to ensure users received the best experience and took advantage of available hardware when needed. This would include times when the smartphone may be used for gaming. OnePlus says they are planning to continue using this model of adjustments to their processor operations, but they would put an end to their system doing the same thing when benchmarking apps are detected. The XDA Developers note that the OnePlus solution creates an artificial limitation for users who have to rely on the operating system being updated for new, processor-intensive applications. The preferred method for triggering resources ramping up their operations is to have the system respond to increased demand rather than just an app’s name.
Testing revealed the Meizu Pro 6 is utilizing a similar method for turning on the big cores in their chips – looking for specific app names. Perhaps a little bit to their credit, Meizu at least provides a user-facing option to place the device in “performance mode” or “balanced mode.” The XDA Developers indicate they still think the performance mode specifically looks for benchmarking apps. Further, the performance mode does not appear to be a setting that can be used for normal day-to-day use, even by heavy gamers.
Hopefully these instances of manufacturers ratcheting up performance in response to benchmarking apps is not a trend that is returning to the market. Benchmark scores nearly became a useless tool for buyers during the last round of cheating and it would be unfortunate to return to that state.
source: XDA Developers