Compatibility Testing Device Selection: A 6-Level Guide (2026)

Source: TesterHome Community

Background

Test design should always depend on what you are actually testing. This is a well‑known principle in software testing.

For terminal‑side testing, compatibility testing is essential. However, the number of phone models on the market is huge. Covering all of them is impractical.

When designing a terminal compatibility or performance test plan for mobile apps, testers often face these challenges:

1. What do different phone parameters actually affect? Which parameters matter most?
2. Which devices should you pick for a given test? How many? Are some devices mandatory?
3. If you finish testing on a set of devices, does that mean your compatibility testing is sufficient?
4. How do you define low‑end, mid‑range, and high‑end devices?

This article answers these questions step by step.

Level 1: How Device Parameters Affect Software and Performance

A phone’s compatibility and performance are constrained by its hardware specifications.

Key parameters include:

• CPU
• Memory (RAM)
• Screen type (notch, punch‑hole, etc.)
• Screen resolution
• Refresh rate
• Manufacturer ROM

What each parameter impacts

Understanding the relationship between device parameters and your app’s test points is the first step in selecting the right compatibility test devices.

Level 2: Leverage Manufacturer Insights

Beyond raw specifications, you need to consider:

• Technical trends
• New features
• The balance between software and hardware

You can gather valuable insights from:

• New device launches – Flagship models have high user share; concept devices show future trends.
• Landmark models – For example, the iPhone X (first mainstream notch screen). Testing one such device often covers an entire category.
• Signature flagships – High performance, recent OS, and typically high user share.
• Real‑world test cases – For instance, iOS 15 changed its boundary correction method, which triggered assertion failures and app crashes in some cases.

Using these channels saves time, keeps you informed, and makes device selection more efficient.

Level 3: Follow Market Share Data

Good test design prioritizes devices with higher market share. More users mean higher risk if something breaks.

Track these four types of share data:

1. Overall device market share – For Android, check Android Studio.
2. Your app’s device share – Available from most analytics platforms.
3. Share of problem‑prone devices – Which devices appear most often in bug reports? These have higher risk.
4. User feedback share – Devices that generate more feedback (suggestions, complaints, bug reports) need more testing attention.

Level 4: Tier Your Devices

Phone performance is largely determined by CPU and memory. Based on these, you can group devices into three tiers.

Tier definitions

How to handle older flagships

A flagship from two years ago becomes a mid‑range device today. CPUs with similar performance should also be classified as mid‑range.

Note: There is no official standard for these tiers. The above classification is based on real‑world testing experience.

Once tiered, combine with market share data to select high‑share, high‑risk devices from each tier.

Level 5: Include Special Cases

Manufacturers often introduce exotic screen types and custom OS behaviors. These can cause unexpected bugs.

Do not miss:

• Foldable phones – Unique screen and OS customization may cause display or interaction issues.
• Android / iOS versions – The same phone model on different OS versions can behave differently. For example, an iPhone on iOS 14 may work perfectly while the same model on iOS 15 fails.

Explicitly include these special devices in your selection process.

Level 6: Always Consider the System Under Test (SUT)

All analysis must connect to real testing practices. Every test plan must be grounded in actual test scenarios.

Examples

Connecting scenarios to parameters

Example: a “send gift” feature. Sending a gift triggers an animation.

This connects a business scenario (sending a gift) to an impacted parameter (animation performance).Only when you make these connections can you select the optimal set of compatibility test devices.

Automating Device Selection

Manually writing test plans for every release is inefficient. A system can automate the process.

System design concept

The system needs two things:

1. A way to combine test scenarios with device parameters
2. A complete device parameter database

Algorithm logic

1. For a given scenario, identify the key parameters that impact it.
2. Match those parameters against the device database.
3. Group devices with identical key parameters into equivalence classes.
4. Flag frequently problematic devices as high‑risk.
5. Rank all grouped devices by market share.

The result is a precise, risk‑aware, and data‑driven device recommendation.

Algorithm flow

Data collection

Gather device specifications, app‑specific usage share, and high‑bug‑frequency devices from your bug tracker.

Data maintenance

Keep the database updated with new devices and rankings.

Scenario maintenance

Allow manual addition of new test scenarios as needs grow.

Recommendation logic

• Read selected scenario → fetch high‑impact parameters
• Match parameters → group identical devices
• Sort each group by market share (highest first)
• Identify top‑share devices from major brands (Huawei, Xiaomi, Oppo, Vivo) → mark as preferred
• Mark scenario‑specific high‑bug devices as high‑risk → strongly recommend

Display

Show groups with clear labels, e.g., “Notch Screen + 1080P”.

Example

Take Scenario A. The system knows that parameters b and c matter most.

It filters the database for devices where b AND c match.

All such devices form one equivalence class → one test group.

The group name is a combination of b and c.

Special handling for screen resolution

Do not match exact values like 2400x1080. Use categories instead: 1080p, 2K, 720p.

Reason: 2400x1080 and 2216x1080 are both 1080p screens. Exact matching would create unnecessarily fragmented groups.

Summary

By following this layered approach – from parameter impact to market share, device tiering, special cases, and finally automation – you can design smart, efficient, and highly effective compatibility test plans.

Compatibility bugs will have nowhere left to hide.