WeTest Case Study: A Battle Royale Mobile Game

This case provides a best-practice model for the mobile game industry, demonstrating how to update QA management using modern technologies and automated tools.

Introduction

This case is about one of the most popular Battle Royale mobile games worldwide.

During the past crisis season, its development team decided to update its strategy on Total Quality Assurance Optimization.

Before the update, its QA strategy focused primarily on functional and regression testing, depended mainly on existing defects and reports, and only focus on application-level testing.

As a result, it has a user crash rate higher than 7%, its client performance issues affected more than 2/3 (two-third) of total users, which directly leads to low ratings on the Apple Store.

To boost its game performance, we update its QA strategy through three dimensions:

During R&D and Testing, we conducted daily build automated functionality and compatibility testing. By using modern technologies and automated testing tools to conduct comprehensive testing, we positioned and fixed more than 1,000 R&D issues.

Secondly, before major releases, we launched advanced compatibility testing and server performance testing to conduct predictive and prescriptive testing and quality assurance.

After the release of each new update, real-time performance data and crash information are captured to continuously refine long-term user experiences.

Automated testing & AI testing

During the updated QA process, automated testing and AI testing plays an important role to reduce the cost and improve efficiency.

To implement Automated testing, we will first start the game, then enter the game scenario automatically through UI-Automator. The weight file will be used to traverse game scenarios. During the process, issues can be automatically found (through game function Bug detection, game scenario coverage test, and game blurred screen and lagging detection). Finally, an in-depth test report will be generated. The typical issues found included Crash, ANR, process exception, black and white screens, and blurred screens. Moreover, we used performance data, logs, and screenshots to do comprehensive analysis of these issues.

Overall Quality Management

Not only did we update the technology and tools of the testing services, but we also updated the overall testing life-cycle for this mobile game.

Compatibility and Functionality Testing starts early during the R&D phase, along with network testing, server/client performance testing, and game security testing. For the game design side, we added playability evaluation to analyze the market and prevent bad game reputation.

After the game has launched, we iteratively conduct security testing, user crash real-time monitoring, compatibility testing, and social listening to enhance the user gaming experience.

Result

The general picture for this case can be concluded into four parts:

· User research and gameplay guarantees

· Player sentiment management

· Game quality in-depth optimization

· Live operation management

After we helped this mobile game to optimize its QA strategy, the number of its daily active users was increased by 35%.