Usability Testing: What It Is, How To Run It And 40 Questions To Ask

Short answer: Usability testing is a research method where you watch real users try to complete realistic tasks with a product, so you can see where they succeed, struggle or fail. A facilitator (or a testing platform) gives each participant tasks, observes their behavior and asks follow-up questions. For qualitative testing, Nielsen Norman Group recommends about 5 users per round, repeated over several rounds, while quantitative benchmarking needs at least 20.

Usability testing is the most direct way to find out whether people can actually use what your team designed. It works on anything from a Figma prototype to a live SaaS app.

This guide covers what usability testing is (and is not), the main types, an eight-step process for running a study, the “5 users” rule and its limits, the metrics worth tracking, and a bank of 40 usability testing questions grouped by stage that you can drop straight into your script.

• What Is Usability Testing?
• Types Of Usability Testing
• How To Conduct Usability Testing In 8 Steps
• How Many Users Do You Need?
• 40 Usability Testing Questions To Ask
• Usability Testing Metrics
• Common Usability Testing Mistakes
• Usability Testing Tools
• When To Hire A UX Research Agency
• Frequently Asked Questions
• The Short Version

What Is Usability Testing?

Usability testing is an evaluation method in which representative users attempt specific tasks with a product while a researcher observes. The goal is to uncover problems in the design: confusing labels, hidden controls, broken flows, unclear error messages, and anything else that stops people from reaching their goal.

Three elements define a usability test:

• Participants who resemble your real or intended users, not colleagues or stakeholders.
• Tasks that reflect what those users actually need to do, written as realistic scenarios.
• Observation of behavior, ideally with participants thinking aloud so you hear their reasoning as they go.

Usability testing is behavioral. It measures what people do, not what they say they would do. That is what separates it from surveys, interviews and focus groups, which capture opinions and attitudes. It also differs from expert reviews such as a heuristic evaluation, where UX specialists inspect an interface against design principles without involving users. The two work well together: an expert review catches obvious issues cheaply, then testing confirms which problems actually trip people up.

If you are deciding where usability testing fits among interviews, card sorting, diary studies and analytics, our guide to UX research methods maps each method to the questions it answers.

Types Of Usability Testing

Most studies combine choices along a few axes: whether a facilitator is present, where the session happens, and whether you are after insights or numbers. The table below summarizes the main options.

Type	How It Works	Best For	Trade-Offs
Moderated	A facilitator runs the session live, gives tasks and asks follow-up questions	Complex flows, early prototypes, understanding the “why”	Slower, needs a skilled moderator, fewer sessions per day
Unmoderated	Participants complete tasks alone on a testing platform that records screen and voice	Simple tasks, fast turnaround, larger samples	No follow-up questions, more low-quality sessions to discard
Remote	Sessions run over video call or a testing platform from the participant’s own device	Distributed users, lower cost, realistic environments	Less control over setup, harder to read body language
In-person	Participant and facilitator share a room or lab	Physical products, kiosks, mobile hardware, sensitive topics	Recruiting is limited by geography, higher cost per session
Qualitative	Small sample, focus on finding and understanding problems	Iterative design, discovering issues	No statistically reliable numbers
Quantitative	Larger sample, focus on metrics such as success rate and time on task	Benchmarking, comparing releases or competitors	Needs 20 or more users per group, less insight into causes
Guerrilla	Quick, informal tests with people in a cafe, event or office lobby	Early concepts, broad consumer products, tiny budgets	Participants may not match your audience; weak for niche B2B tools
A/B testing (not usability testing)	Live traffic is split between two versions and outcomes are compared	Choosing between variants at scale	Shows which version wins, not why; needs significant traffic

Moderated Vs Unmoderated Usability Testing

Choose moderated testing when tasks are complex, the product is early, or you need to probe reasoning. Choose unmoderated testing when tasks are simple and self-explanatory, you need results within a day or two, or you want a larger sample for metrics.

Why A/B Testing Is Not Usability Testing

An A/B test tells you which variant converted better with live traffic. It does not show anyone struggling or explain why one version won. Use usability testing to generate and refine design options, then A/B testing to choose between strong candidates at scale.

How To Conduct Usability Testing In 8 Steps

The process below assumes a moderated remote study. Unmoderated studies follow the same steps, with the script turned into on-screen instructions.

1. Define Your Goals

Write down what you need to learn and what decision the results will inform. “Can new admins invite their team without help?” is a usable goal. “See what users think of the app” is not. Keep it to 2 to 4 research questions per round, and agree on them with product and design leads before recruiting anyone.

2. Write Realistic Tasks

Turn each goal into a scenario that gives context and a clear end point without revealing the answer. Avoid using the exact words that appear in the interface, because participants will simply scan for them.

• Weak: “Click Settings and add a user.”
• Better: “A new colleague, Sam, starts on Monday. Give Sam access to your workspace so they can view reports but not change billing.”

Plan 4 to 7 tasks for a 45 to 60 minute session, ordered the way a real user would encounter them. Define what counts as success for each task before testing begins.

3. Recruit The Right Participants

Recruit people who match your target users on the attributes that affect behavior: role, experience with similar tools, company size, device and frequency of use. Sources include your customer list, in-app intercepts, platform recruiting panels and specialist B2B recruiters. Filter with a screener survey, pay fair incentives, and over-recruit by one or two people to cover no-shows.

4. Write A Usability Testing Script

A usability testing script keeps sessions consistent across participants and moderators. A standard script has five parts:

1. Welcome and consent: introduce yourself, explain the purpose, confirm recording consent, and state clearly that you are testing the product, not the person.
2. Think-aloud instructions: ask the participant to say what they are looking at, trying to do and expecting as they go.
3. Warm-up questions: a few background questions to build rapport and add context.
4. Tasks: each scenario read aloud or pasted into chat, followed by post-task questions.
5. Wrap-up: post-test questions, a questionnaire such as SUS, thanks and incentive details.

5. Run A Pilot Session

Run one full session with a colleague or a friendly user before the real study. Pilots catch broken prototype links, tasks that are ambiguous or impossible, timing problems and recording issues. Fix everything you find and update the script.

6. Run The Sessions

During each session, the moderator’s job is to stay neutral and quiet. Read tasks verbatim, let silences run, and answer questions with questions (“What would you expect to happen?”). Have one or two observers take timestamped notes in a shared template, and leave 15 to 30 minutes between sessions to reset and capture first impressions.

7. Analyze The Findings

Gather observations from every session into one place, such as a spreadsheet or a digital whiteboard. Group them by task and by theme, then count how many participants hit each problem. Rate each issue by severity, for example:

• Critical: stops users from completing the task.
• Serious: causes major delay, errors or frustration but users recover.
• Minor: noticeable friction with little effect on outcome.
• Cosmetic: polish issues to fix when time allows.

8. Report And Act

Keep the report short and decision-focused: goals, method, participants, top findings ranked by severity, short video clips of key moments, and recommended next steps. Then feed fixes into the backlog and schedule the next round to verify them. Usability testing works best as a loop inside your product design process, not as a one-off event before launch.

How Many Users Do You Need?

The most quoted guidance comes from Jakob Nielsen’s 2000 article Why You Only Need to Test with 5 Users. Based on a model in which each user reveals roughly 31% of the usability problems in a design, testing with 5 users uncovers about 85% of the problems. After that, each additional user mostly repeats what you have already seen.

The rule is often misread as “5 users is always enough.” The original guidance comes with important caveats, which NN/g restates in its summary article on how many test users to include:

• Iterate rather than enlarge. Nielsen’s point is that a budget for 15 users is better spent on 3 rounds of 5, fixing problems between rounds, than on one study of 15.
• It applies to qualitative testing. Quantitative studies that report metrics with confidence intervals need at least 20 users, and more for tighter precision.
• Distinct audiences need their own participants. If you serve very different user groups, NN/g suggests 3 to 4 users per group for two groups, and at least 3 per group for three or more.
• Other methods differ. NN/g recommends 15 or more users per group for card sorting and 39 users for stable eyetracking heatmaps.
• The 85% figure is an average from a model. Complex products, very broad audiences or rare edge cases can require more sessions to surface important problems.

In practice: for a qualitative round on one audience, plan 5 or 6 sessions, then test again after fixes. For benchmarking, plan 20 or more per group.

40 Usability Testing Questions To Ask

Good usability testing questions are open, neutral and tied to behavior. Adapt the wording to your product and pick only the questions that serve your goals; no session needs all 40.

Screener Questions (1 To 7)

Use these in a recruiting survey to confirm participants match your target users. Avoid revealing the “right” answer, or people will select it to qualify.

1. Which of the following best describes your current job role?
2. How many people work at your company?
3. Which of these tools have you used in the past 3 months? (Include your product and competitors among several options.)
4. How often do you do [the core activity your product supports]?
5. Who in your organization decides which [category] tools to buy?
6. Which devices do you mainly use for work tasks like this?
7. Have you taken part in a product research study in the past 6 months?

Pre-Test Questions (8 To 13)

Ask these at the start of the session to build rapport and gather context you can refer back to during tasks.

8. Can you tell me a bit about your role and a typical workday?
9. Walk me through the last time you [did the core task]. What did you use?
10. What is the most frustrating part of that process today?
11. What tools or workarounds do you rely on to get it done?
12. What would a better way of doing this look like for you?
13. Have you used [product] before? If so, what for?

Task Prompts And In-Task Questions (14 To 23)

These are neutral prompts a moderator can use while a participant works through a task. They encourage thinking aloud without leading.

14. Looking at this page, what do you think it is for?
15. Where would you start to complete this task?
16. What are you looking for right now?
17. What do you expect to happen when you click that?
18. Is that what you expected? What is different?
19. What does this label or term mean to you?
20. You paused there. What were you thinking?
21. What would you do next if I were not here?
22. How would you know if you had finished the task?
23. Is there anything on this screen you would ignore?

Post-Task Questions (24 To 31)

Ask these right after each task, while the experience is fresh. Pair them with the Single Ease Question covered in the metrics section.

24. Overall, how difficult or easy was that task to complete?
25. What, if anything, was confusing about that task?
26. Was anything missing that you expected to find?
27. How confident are you that you completed the task correctly?
28. Was there a point where you felt stuck? What happened?
29. How does this compare to how you do it today?
30. If you could change one thing about that flow, what would it be?
31. How long did that feel like it took, compared with what you expected?

Post-Test Questions (32 To 40)

Close the session with broader reflections, then the SUS questionnaire if you are tracking it.

32. How would you describe this product to a colleague?
33. What did you like most about what you saw today?
34. What was the most frustrating part of the session?
35. Which part of the product would you use most often, and why?
36. Was anything missing that would stop you from using this at work?
37. What questions do you still have about how it works?
38. How likely would you be to use this instead of your current approach, and what would need to change?
39. If you had a magic wand, what would you change first?
40. Is there anything else you would like to share that we did not ask about?

Usability Testing Metrics

Qualitative rounds produce findings, not statistics, but a few simple metrics help you compare tasks and track progress across releases. With small samples, report raw counts (“4 of 5 participants completed the task”) rather than percentages that imply false precision.

Metric	What It Measures	How To Capture It	Reference Point
Task success rate	Share of participants who complete a task	Define success criteria per task before testing; record pass, partial or fail	Compare against your own previous rounds or competitors
Time on task	How long successful participants take	Timestamps from recordings or platform logs	Most useful in quantitative studies; think-aloud slows people down
Error rate	Mistakes made during a task, such as wrong clicks or invalid inputs	Define error types upfront and tally per task	Track trends between iterations
SEQ (Single Ease Question)	Perceived task difficulty on a 7-point scale	Ask immediately after each task	MeasuringU reports an average of about 5.5
SUS (System Usability Scale)	Overall perceived usability on a 0 to 100 score	10-item questionnaire at the end of the session	MeasuringU reports an average of 68

Task Success Rate

NN/g calls success rate the simplest usability metric. You can score partial success (for example, completed with minor or major issues), but NN/g cautions against turning those levels into numbers and averaging them, because the levels are ordinal labels, not measurements.

SEQ And SUS

The Single Ease Question asks “Overall, how difficult or easy was the task to complete?” on a 1 to 7 scale. The System Usability Scale, created by John Brooke in 1986, uses 10 statements rated on a 5-point agreement scale, converted to a score from 0 to 100. A SUS score is not a percentage: 68 is roughly average, not a failing grade.

Common Usability Testing Mistakes

Most failed studies fail for avoidable reasons. Watch for these:

• Testing with the wrong people. Colleagues, friends and power users who know the product will not reveal what new or typical users struggle with.
• Leading tasks. Task wording that mirrors interface labels turns the test into a word search.
• Helping too soon. Rescuing a participant the moment they hesitate hides the exact problem you came to find.
• Skipping the pilot. Broken prototypes and ambiguous tasks waste real sessions.
• Treating small samples as statistics. “80% success” from 5 people is 4 people. Report counts.
• Testing too late. Testing only before launch leaves no time to fix what you find. Test prototypes early, even rough ones.
• Producing a report nobody acts on. Without severity ratings, owners and follow-up rounds, findings sit in a slide deck.

Usability Testing Tools

A video call tool, a clickable prototype and a shared notes document are enough for a useful moderated test. As testing becomes routine, dedicated tools save time:

• All-in-one research platforms: recruiting panels, unmoderated and moderated testing, recordings and analysis in one place. UserTesting is a well-known example.
• Prototype and unmoderated testing tools: connect a Figma prototype and collect task paths, success rates and heatmaps at scale. Maze and Lyssna (formerly UsabilityHub) are examples.
• Moderated session and recording tools: purpose-built for live remote sessions with observer rooms and timestamped notes. Lookback is an example.
• Participant recruiting services: panels and B2B recruiting specialists that find and pay participants for you.

Features and pricing change often, so compare current plans on each vendor’s own site and run a small trial study before committing.

When To Hire A UX Research Agency

Many teams run usability testing in house successfully. Outside help makes sense in a few situations:

• You have no dedicated researcher and designers are too close to the work to moderate neutrally.
• Recruiting is hard, for example clinicians, IT buyers, finance leaders or participants in several countries and languages.
• The stakes are high, such as a redesign, a regulated workflow or a benchmark study that will guide a large investment.
• You need quantitative rigor: larger samples, competitive benchmarking and proper statistical analysis.

If you are shortlisting partners, our list of the top UX research and usability testing agencies compares firms that run moderated and unmoderated studies. If you want an expert review of an existing product before investing in user testing, see the top UX audit companies. And if testing reveals deeper design problems, the top UX design agencies can take findings through to redesign.

Frequently Asked Questions

What is usability testing in simple terms?

Usability testing means watching real users try to do real tasks with your product so you can see where it confuses or blocks them. It focuses on behavior rather than opinions. The output is a prioritized list of problems and fixes.

How many users do you need for usability testing?

For qualitative testing with one audience, Nielsen Norman Group recommends about 5 users per round, with several rounds and fixes in between. Distinct user groups each need their own participants. Quantitative studies that report metrics need at least 20 users per group.

What is the difference between moderated and unmoderated usability testing?

In moderated testing, a facilitator runs each session live and can ask follow-up questions. In unmoderated testing, participants complete tasks on their own through a platform that records them. Moderated testing gives deeper insight; unmoderated testing is faster and scales to larger samples.

How do you run remote usability testing?

For moderated remote testing, share a prototype or live product link over a video call, ask the participant to share their screen, and record with consent. For unmoderated remote testing, set up tasks on a testing platform and send participants a link. Either way, pilot it first.

What should a usability testing script include?

A usability testing script includes a welcome and consent section, think-aloud instructions, warm-up questions, task scenarios with post-task questions, and closing questions plus any questionnaire such as SUS. Reading it verbatim keeps sessions consistent. Update it after the pilot.

How long should a usability test session last?

Moderated sessions usually run 30 to 60 minutes, which fits 4 to 7 tasks plus questions. Unmoderated tests are usually shorter, often 10 to 20 minutes.

The Short Version

Usability testing is watching representative users attempt realistic tasks so you can find and fix the problems that stop them. Define 2 to 4 goals, write neutral task scenarios, recruit the right people, script and pilot the session, then analyze findings by severity and report with video clips.

Test about 5 users per audience per qualitative round and iterate, or 20 or more for quantitative benchmarks. Track task success, time on task, errors, SEQ and SUS where they help, and use the 40 questions above as a starting bank. Bring in an agency when recruiting, rigor or capacity is beyond your team.