The phones delisted were the P20, P20 Pro, Nova 3 and the Honor Play.

“After testing the devices in our own lab and confirming that they breach our rules, we have decided to delist the affected models and remove them from our performance rankings,” the company said in a statement.

For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI; however, when an unlabeled version of the benchmark test was run, the phones were unable to recognize it and, as a result, displayed lower performances.

In other words, the phones aren’t so smart after all.

Huawei is not the first company caught overstepping these rules. Samsung did as well in 2013, and ironically the results of these benchmark tests actually mean little in terms of overall general performance of the phone.

While they can point to how a phone may perform during heavy stress, average performance is still best discovered through individual testing.

Huawei did not immediately return a request for comment.

Update: In a comment late Thursday night, a Huawei spokesperson told TechCrunch:

Huawei always prioritizes the user experience rather than pursuing high benchmark scores – especially since there isn’t a direct connection between smartphone benchmarks and user experiences. Huawei smartphones use advanced technologies such as AI to optimize the performance of hardware, including the CPU, GPU and NPU.

In normal benchmarking scenarios, once Huawei’s software recognizes a benchmarking application, it intelligently adapts to “Performance Mode” and delivers optimum performance. Huawei is planning to provide users with access to “Performance Mode” so they can use the maximum power of their device when they need to.

Huawei – as the industry leader – is willing to work with partners to find the best benchmarking standards that can accurately evaluate the user experience.