What is Compatibility Testing?

Compatibility Testing is a game testing method to ensure that the game runs successfully on different hardware, operating systems, or mobile devices.

Compatibility issues examples:* (Character misplacement & Blurred Screens)*

Android Fragmentation

Nowadays, compatibility testing for mobile games faces big challenges as there are a massive number of different Android OS versions and models. There are now over 1,300 Android brands that have produced over 24,000 distinct Android devices.

The image below is taken from OpenSignal’s July 2013 report on Android fragmentation, which indicates the number of different Android devices that downloaded their app in the first half of 2013. There were already a large number of devices in 2013. Now in 2021, the number can only be much larger.

source: https://www.opensignal.com/market-insights

Android Fragmentation indicates the need to test an increasing number of devices and versions during the compatibility testing for mobile games.

In this case, efficient and scientific selection of test device models becomes very important as the basis for compatibility testing.

Efficient selection of device models

Although it’s always better to have as many phone models tested as possible, it is not cost-effective to test all the available models. To boost efficiency, we need to determine the test models more objectively and accurately based on user insight and big data.

In WeTest, several different solutions are differentiated with the number of phone models covered.

The TOP100 models cover almost all common Android versions, resolutions ratio, memory, manufacturers, CPUs, and GPUs. The TOP300 models cover about 87% of Chinese game users. For PUBG mobile, WeTest used the TOP500 models, which cover more than 90% of game users nationwide. Lastly, the TOP1000 models can represent 80% of the global users.

Efficiently test the game scenarios

After determining the test models, we need to efficiently test the game scenarios and automatically find the issues.

Automated & AI testing are often used to identify issues in specific game scenarios. The method uses game AI training flow, and the weight file will be generated.

First, start the game.

Then, enter the game scenarios automatically through UI-Automator.

The weight file will be used to traverse more game scenarios. During the process, issues can be automatically found.

Finally, the depth test report will be generated. The potential issues that could be found include crash, ANR (Application Not Responding), process exception, black and white screens, blurred screens, etc. Performance data, logs, and screenshots are also used to conduct a more comprehensive analysis.

Real Devices in cloud lab

For the issues found during the testing, real devices are needed to locate the issues more accurately and efficiently. WeTest uses thousands of real devices in its cloud lab. Websites and phones are used to easily monitor and control these cloud devices, with low latency (less than 50 milliseconds). It also has real-time log and sound output and uses ADB commands for debugging. With its intelligent scheduling strategy, data center self-recovery, efficient remote operations, and automated testing framework, real device testing can ensure higher stability, availability, confidentiality for the overall solution.

For more information about Compatibility Testing, please visit https://www.wetest.net/overview?product_id=65