UX Testing Methods: Usability, Expert & Simulated User Testing | Guide

Source: TesterHome Community

1. Background

As digital transformation continues to deepen, the importance of user experience (UX) is becoming increasingly evident.

UX is gradually becoming a core competitive advantage for software products.

As a critical quality assurance checkpoint, UX testing is an indispensable part of system testing.

Currently, in-house testing mainly focuses on:

• Unit integration testing
• Functional testing
• Performance testing
• Security testing

A key priority is ensuring that all critical, severe, and general defects are resolved before system launch.

However, existing testing processes do not define clear criteria from a UX perspective.

To improve user satisfaction and refine our in-house UX testing methodology, the Chengdu R&D Department initiated research on UX testing.

2. The Concept of User Experience

The ISO 9241-210 standard defines user experience as:

“A person’s perceptions and responses that result from the use or anticipated use of a product, system or service.”

In practice, UX includes all physiological, sensory, and psychological experiences a user has when using a product or service, combined with their prior experiences.

This includes:

• Emotions, beliefs, and preferences
• Perceptions and impressions
• Physiological and psychological reactions
• Behaviors and achievements

2.1 Factors Influencing UX

UX is influenced by three main factors:

3. UX Testing Methods

3.1 Usability Testing

Definition

Testers write and execute test cases based on a usability testing indicator system, using precise expected results to identify UX flaws.Prerequisite

UX must have been considered during the product design phase.Goal

Compare the final product against the prototype design to confirm whether UX requirements are met.

3.1.1 Usability Testing Indicator System

The system is divided into three levels:

Level 1 Indicators Explained

3.2 Expert Testing

Definition

UX designers, interaction designers, or other professionals with relevant experience directly test the product for UX issues.Method

They follow Nielsen’s 10 heuristics (referred to here as Expert Testing Principles) and refer to a list of function points.

Key feature

No test cases need to be written.

3.3 Simulated User Testing

Definition

Testers design UX testing tasks within a defined scope, invite simulated users to perform these tasks, and collect execution data.Focus areas

• Metrics during user operation
• Problems raised by users

What it reveals

• What users focus on during actual use
• Where they encounter difficulties
• Post-task feelings and overall satisfaction

Goal

Gain a comprehensive understanding of UX issues real users face.

4. Practical Outcomes

The three UX testing methods were applied to three systems:

• Task Management System
• New Community System
• Cloud Notes System

A series of documents were produced as a result.

4.1 Example: Task Management System

As these issues were progressively fixed, the system’s UX improved significantly.

Specific example

• The “Import Task Template” button originally had:Unclear meaning
• Illogical placement
• Inconspicuous design style

As a result, users:

• Did not understand the button’s purpose
• Mistakenly associated it with the page title
• Often ignored it entirely

After fixes, the button’s function became significantly clearer and easier to understand.

5. Summary and Reflections

5.1 Usability Testing

• Standard method
• Best when design specifications are comprehensive and UX is considered from the start
• Focuses on comparing the final system against the prototype
• Can identify most UX issues effectively

5.2 Expert Testing and Simulated User Testing

• Non-standard methods
• Can be introduced during:
  • System prototype design phase, or
  • After functional development is complete
• Recommended when design specifications are incomplete or UX was not initially considered

5.3 When to Use Simulated User Testing

5.4 Final Thoughts

Using these three methods effectively drives product UX improvement.

However, testing alone is not enough.

Improving UX requires a joint effort from:

• Business teams
• Design teams
• Development teams
• Other related teams

Further UX enhancements should be explored through:

• User research
• Usability testing
• Design system construction
• Related activities