Foreword

In 2026, AI Agent and cloud-native technologies are reconstructing the entire software development process, and the software testing industry is ushering in a key transformation from "manual back-up" to "intelligent front-end". Data shows that the average monthly failure rate of traditional test scripts is still as high as 25%, and maintenance costs account for more than 60% of the total testing workload. However, AI-driven testing solutions have achieved several times improvement in efficiency and have become the core choice for quality assurance in industries such as finance and automobiles.

 

Facing the new paradigm of "human-machine collaboration", whether you are a novice who wants to get started or a practitioner who is seeking to upgrade his skills, you need a knowledge system that takes into account basic logic and cutting-edge trends.

 

To this end, the TesterHome community has launched a series of articles called "Advanced Testing Quality", which starts from the core understanding of testing, gradually deepens into process specifications, tool operations, and special practices, and finally connects to cutting-edge fields such as AI testing and cloud native testing. Through systematic content output and practical case dismantling, it helps everyone build testing capabilities that adapt to industry changes. (This series will continue to be updated, so stay tuned!)

 

Introduction: Quality Model is the "Measurement Ruler" of Testing

In software testing work, "how to determine whether software quality is qualified" is the core issue. If there is no unified quality measurement standard, the testing work will fall into a disorderly state of "finding bugs based on experience", making it difficult to fully cover quality risks. The core principles of testing provide methodological guidance for testing work, while the quality model is a "metric ruler" for quantitatively evaluating software quality.

 

With the development of technologies such as AI and cloud native, traditional quality models are facing new adaptation challenges, and intelligent quality assessment systems have emerged. This article will start with the classic ISO25010 quality model, dismantle the core quality characteristics and testing points, and then combine it with cutting-edge technology trends to explore the changes and practical directions of quality assessment in the intelligent era, helping everyone to establish a scientific testing thinking of "principle guidance + model measurement".

 

1. In-depth Interpretation and Implementation Scenarios of Core Principles of Testing

In the first article, we initially sorted out the six core principles of software testing. These principles are not abstract theories, but practical guidelines throughout the entire testing process. This section will combine specific scenarios to deeply interpret the core connotation and implementation points of each principle:

 

(1) Test as Early as Possible and Participate in the Whole Process

- Key to Implementation:

The core value of early testing in the demand stage is to "reduce the cost of defect repair". Data shows that if defects introduced in the demand stage are discovered only after they go online, the repair cost is more than 100 times that of repairing in the demand stage.

- Focus Scenarios:

  1. Requirements review process: Testers need to intervene in advance, identify ambiguity points (such as vague expressions such as "good user experience") through "demand testability analysis", and promote the requirements document to supplement quantitative indicators (such as "page loading time ≤ 2 seconds");

  2. Design stage: Participate in architecture design review and predict quality risks that may be caused by technology selection (such as interface consistency risks of microservice architecture).

- Intelligent Era Upgrade:

Natural language processing (NLP) tools can automatically parse requirement documents, identify untestable items and give optimization suggestions, further improving the efficiency of front-end quality control.

 

(2) Risk-Driven Testing

- Key to Implementation: Scientifically dividing risk priorities. Exhaustive testing is not feasible, so it is necessary to focus on high-risk modules.

- Implementation Method: Divide priorities through the "risk matrix": use two dimensions of "impact degree" (such as core trading functions vs. auxiliary functions) and "probability of occurrence" (such as high-frequency operation scenarios vs. low-frequency scenarios) to classify test objects into three risk levels: high, medium, and low.

- Example: The "transfer function" of a financial APP is a high-risk module with "high impact + high probability" and requires the most investment in testing resources; the "historical transaction export function" is a low-risk module with "low impact + low probability" and can appropriately simplify testing.

- Intelligent Era Upgrade: The intelligent testing platform can automatically generate a risk matrix by analyzing historical bug data and user operation logs, accurately locate high-risk modules, and improve the efficiency of testing resource allocation.

 

(3) Defect Cluster Effect

- Key to Implementation: Focus on in-depth testing of high-frequency defective modules.

- Implementation Method: Establish a "defect statistical analysis mechanism" to regularly summarize defects discovered during testing and locate modules with concentrated defects (e.g., more than 5 defects were found in the "order payment module" of an e-commerce APP in a row). For such modules:

  1. In addition to executing regular test cases, supplement "abnormal scenario testing" (such as network interruption, boundary conditions of abnormal data);

  2. Use static code analysis tools (such as SonarQube) to troubleshoot potential code-level problems (such as code redundancy, logic loopholes) when necessary;

  3. Analyze the root causes of concentrated defects (such as developers' technical shortcomings, frequent requirement changes) and promote the team to solve them fundamentally.

 

(4) Independence and Objectivity of Testing

- Key to Implementation: Separation of responsibilities between testing and development.

- Implementation Method: Avoid "developer self-testing and self-verification", and adopt the model of "development self-testing + independent verification by the testing team":

  1. After writing code, developers first conduct unit testing and integration testing to ensure the code is runnable;

  2. The testing team conducts third-party verification based on independent testing environments and test cases.

- Enhanced Suggestion: For core business systems, introduce an "independent testing team" (not directly affiliated with the development team) to further ensure objectivity. Test case design should adopt "reverse thinking" (e.g., development focuses on normal processes, testing focuses on abnormal processes) to avoid being affected by development ideas.

 

(5) Testing Should Trace the Requirements

- Key to Implementation: Two-way traceability of use cases and requirements.

- Implementation Method: Establish a two-way traceability system of "requirements-use cases-defects":

  1. Each test case is associated with the corresponding requirement point;

  2. Each defect is traced back to the corresponding use case and requirement.

-Example: A requirement point "Supports login with mobile phone number" needs to be associated with test cases such as "Login with correct mobile phone number", "Login with wrong mobile phone number", "Login with empty mobile phone number"; if the defect "Wrong mobile phone number can log in" is found, it must be clearly associated with the corresponding use case and requirement.

-Intelligent Era Upgrade: Intelligent test management tools (such as TestRail) can automatically implement two-way traceability, count requirement coverage in real time, and avoid missing requirements.

 

(6) Human-Machine Collaboration, Complementary Advantages

-Key to Implementation: Reasonable division of labor between AI and humans.

-Division of Labor Principle:

  1. AI is responsible for repetitive and mechanical testing work (such as execution of regression test scripts, generation of a large number of basic use cases);

  2. Humans are responsible for creative work (such as design of complex scenarios, in-depth defect analysis, user experience evaluation).

-Example: For the "product list loading" function of an e-commerce APP, AI can automatically perform compatibility tests on 100 different models; subjective experience tests such as "accuracy of product recommendation algorithms" require manual evaluation based on user needs.

 

2. Classic Quality Model: Core Content and Test Application of ISO25010

A quality model is a standard system for quantitatively evaluating software quality. ISO25010 is a classic model commonly used in the industry, dividing software quality into two dimensions: "product quality" (8 characteristics) and "use quality" (4 characteristics). This section focuses on dismantling the 8 core characteristics of product quality (the focus of testing work) and explains their application methods in combination with specific test scenarios:

 

(1) Functionality

-Core Definition: The ability of software to implement established functions.

-Core Sub-characteristics: Functional completeness (whether all established requirements are covered), functional correctness (whether execution results meet expectations), functional appropriateness (whether functions adapt to user needs).

-Test Scenario Example: For the "order function" of an e-commerce APP, verify:

  1. Completeness: Whether the entire process of "selecting products - submitting orders - completing payment" is covered;

  2. Correctness: Whether the payment amount is consistent with the product amount;

  3. Appropriateness: Whether it supports payment methods commonly used by users.

-Testing Method: Mainly functional testing, covering core scenarios through equivalence classes, boundary values and other use case design methods.

 

(2) Performance Efficiency

-Core Definition: Indicators such as response speed and throughput of software under specified conditions.

-Core Sub-characteristics: Time characteristics (response time), resource utilization efficiency (CPU, memory usage), throughput (number of requests processed per unit time).

-Test Scenario Example: For the "video loading function" of a short video APP, verify:

  1. Time characteristics: "First screen loading time of video under 4G network ≤ 3 seconds";

  2. Resource utilization: "Memory usage ≤ 500MB after playing 10 consecutive videos";

  3. Throughput: "A single server can handle 1,000 playback requests per second during peak hours".

-Testing Method: Mainly performance testing; commonly used tools include JMeter, LoadRunner.

 

(3) Compatibility

-Core Definition: The ability of software to run in different environments.

-Core Sub-characteristics: Hardware compatibility (different devices, chips), software compatibility (different operating systems, browsers), data compatibility (data migration and adaptation of different versions).

-Test Scenario Example:

  1. Office software: Verify stability under operating systems such as Windows 10/11, macOS, and browsers such as Chrome, Edge, Safari;

  2. Mobile APP: Verify display and operation compatibility on devices of different brands (Huawei, Apple, Xiaomi) and screen sizes;

  3. Cloud-native applications: Verify compatibility of containerized deployment under different K8s versions.

 

(4) Usability

-Core Definition: The ease with which users can understand, learn and use software.

-Core Sub-characteristics: Learnability (time for novices to master basic operations), operability (simplicity of operation steps), fault tolerance (prompts and recovery capabilities for user misoperations).

-Test Scenario Example: For the "fund redemption function" of a financial management APP, verify:

  1. Learnability: Whether novices can master the operation within 5 minutes;

  2. Operability: Whether the redemption process does not exceed 3 steps;

  3. Fault tolerance: Whether there is a clear prompt (e.g., "Amount cannot exceed position amount") and support for modification when users enter incorrect amounts.

-Testing Method: Often combined with user research, eye tracking and other methods.

 

(5) Reliability

-Core Definition: The ability of software to continue running under specified conditions.

-Core Sub-characteristics: Maturity (ability to avoid failures), fault tolerance (ability to recover after failures), recoverability (ability to recover after data loss).

-Test Scenario Example: For the reliability test of an enterprise ERP system, verify:

  1. Maturity: Whether it can run continuously for 72 hours without crashing;

  2. Fault tolerance: Whether the database connection can be automatically reconnected after interruption;

  3. Recoverability: Whether data can be restored through backup after system crash.

-Testing Method: Includes stress testing, durability testing, disaster recovery testing.

 

(6) Security

-Core Definition: The ability of software to protect information and data.

-Core Sub-characteristics: Confidentiality (data cannot be accessed without authorization), integrity (data cannot be tampered with), verifiability (operations can be traced).

-Test Scenario Example: For the security test of a social APP, verify:

  1. Confidentiality: Whether user passwords are stored encrypted;

  2. Verifiability: Whether there is a log record after user information modification;

  3. Integrity: Whether unauthorized users can directly obtain others' information through interfaces.

-Testing Method: Includes penetration testing, vulnerability scanning; focuses on common security vulnerabilities such as OWASP Top10.

 

(7) Maintainability

-Core Definition: The ease with which software can be modified and upgraded.

-Core Sub-characteristics: Analyzability (ease of locating defects), modifiability (ease of code modification), stability (whether new defects are introduced after modification).

-Test Scenario Example: After the development team optimizes the code of the "User Management Module", verify:

  1. Stability: Whether new defects are introduced after optimization;

  2. Analyzability: Whether defects can be quickly associated with relevant code when locating.

-Testing Method: Often combined with static code analysis and regression testing.

 

(8) Portability

- Core Definition: The ease with which software can be moved from one environment to another.

-Core Sub-characteristics: Adaptability (ability to migrate without modification), installability (simplicity of installation process), coexistence (ability to run simultaneously with other software).

- Test Scenario Example: When migrating enterprise software from a local server to a cloud server, verify:

  1. Adaptability: Whether it can run without significant code modification;

  2. Installability: Whether the installation steps in the cloud environment are simple;

  3. Coexistence: Whether there is a port conflict with other software on the cloud server.

 

3. Limitations of Traditional Quality Models and Changes in Intelligent Quality Assessment

Traditional quality models such as ISO25010 have played an important role in the quality assessment of traditional software (e.g., stand-alone software, Web applications). However, in the face of emerging software forms such as AI applications, cloud-native systems, and vehicle-mounted intelligent software, their limitations are gradually exposed:

 

(1) Limitations of Traditional Quality Models

1. Difficulty in adapting to dynamic scenarios with static evaluation: Traditional models are mostly based on preset static indicators (e.g., response time, compatibility list), but AI application outputs are uncertain (e.g., recommendation algorithm results change dynamically with user behavior), and cloud-native system elastic scaling leads to dynamic changes in the operating environment, making static indicators difficult to comprehensively evaluate quality;

2. Difficulty in quantitatively assessing AI-specific quality dimensions: Traditional models do not cover core quality dimensions of AI applications such as "algorithmic fairness" (e.g., whether recommendation results discriminate against specific groups) and "interpretability" (e.g., whether the basis for AI decision-making is clear);

3. Assessment efficiency cannot match rapid iteration rhythm: Traditional quality assessment relies mostly on manual statistical analysis, which is time-consuming and difficult to adapt to the "multiple iterations per day" rhythm under the DevOps model.

 

(2) Core Changes in Intelligent Quality Assessment

To address the above limitations, the intelligent quality assessment system has implemented three major changes based on the traditional model:

1. Dynamicization of indicator system: Based on AI and big data technology, real-time collect software operation data (e.g., user operation logs, system performance indicators), and dynamically adjust quality indicator thresholds (e.g., dynamically adjust the "response time" qualification standard based on user concurrency in different periods);

2. Expansion of quality dimensions: Add AI-specific quality dimensions (e.g., algorithm fairness, interpretability, robustness) and cloud-native unique quality dimensions (e.g., elastic scalability, container scheduling efficiency);

3. Automation of evaluation process: Automatically collect indicator data through the intelligent testing platform, generate quality evaluation reports, and even automatically give optimization suggestions based on evaluation results (e.g., "Interface response time is too long, it is recommended to optimize database indexes").

 

(3) Technical Support for Intelligent Quality Assessment

The implementation of intelligent quality assessment relies on three core technologies:

1. Big data collection and analysis technology: Realize real-time collection and correlation analysis of full-link data;

2. Machine learning technology: Build quality prediction models by analyzing historical quality data (e.g., predict the probability of defects in a certain module);

3. Automated testing technology: Realize automatic collection and verification of indicator data (e.g., continuously monitor system performance indicators through automated scripts).

-Example: The intelligent quality assessment platform of an AI recommendation system can collect user click data in real time, analyze the fairness of recommendation results (e.g., whether the recommendation hit rate of users of different age groups is balanced), and automatically generate optimization suggestions.

 

4. Practical Application: How to Combine ISO25010 and Intelligent Assessment to Carry Out Testing

In actual testing work, there is no need to completely abandon the traditional quality model. Instead, the core framework of ISO25010 should be combined with intelligent assessment technology to build an integrated quality assessment system of "basic indicators + dynamic intelligent indicators". The specific practical steps are as follows:

 

Step 1: Build a Basic Quality Indicator System Based on ISO25010

Combine product types, select core quality dimensions from the 8 characteristics of ISO25010, and clarify quantitative indicators.

-Example: For an e-commerce APP, the core dimensions are "functionality", "performance efficiency", "security", and "usability", with corresponding quantitative indicators:

  - Function pass rate ≥ 99.5%;

  - Peak concurrency ≥ 100,000;

  - Zero high-risk security vulnerabilities;

  - Novice operation learning time ≤ 3 minutes.

 

Step 2: Supplement Intelligent Indicators Based on Emerging Technology Characteristics

If the product involves AI, cloud-native and other technologies, supplement corresponding intelligent indicators.

-Example:

  - AI recommendation APP: Add "recommendation accuracy ≥ 85%" and "algorithm fairness deviation ≤ 5%";

  - Cloud-native applications: Add "elastic scaling response time ≤ 10 seconds" and "container availability ≥ 99.99%".

 

Step 3: Use Intelligent Tools to Realize Automatic Collection and Evaluation of Indicators

Select appropriate intelligent testing tools to realize automatic collection, analysis and visualization of indicator data.

-Example:

  - Use JMeter + Prometheus for real-time collection of performance indicators;

  - Use AI testing platform for automatic verification of recommendation algorithm accuracy;

  - Use TestRail for visual display and traceability of quality indicators.

 

Step 4: Establish a Continuous Quality Improvement Mechanism

Regularly analyze quality shortcomings (e.g., "peak concurrency does not meet the standard") based on quality assessment results, and promote the team to optimize the entire process from requirements, design, development, testing, etc.

-Example: In response to insufficient concurrency, push the development team to optimize the database sub-database and table strategy, and the testing team to supplement stress testing of high-concurrency scenarios.

 

Summary: Building Quality Thinking Adapted to the Intelligent Era

The core principles of testing are the "methodology" of testing work; traditional quality models such as ISO25010 are the "basic framework" of quality assessment; intelligent quality assessment technology is an "upgrade tool" to deal with emerging software forms. Mastering the combined application of the three is one of the core abilities of testers in the intelligent era.

 

This article helps everyone establish a scientific quality thinking of "principle guidance + model measurement + intelligent empowerment" by deeply interpreting the implementation scenarios of the core principles of testing, dismantling the core characteristics of the ISO25010 quality model, and discussing the changes and practices of intelligent quality assessment.

 

In the next article, we will focus on "Types and Levels of Software Testing" and explain in detail the key points covering the entire process from unit testing to acceptance testing to help everyone build a complete understanding of the testing level.

(From: TesterHome)