The main goal of usability testing is to answer the question “Can people use this design?” 

In order to find the answer, a common practice is to:

• Put a product or experience in front of someone
• Ask them to perform a set of tasks
• Observe their behavior
• And listen to them talk through what they do as they do it

This can help you understand how real users respond to your product or experience. You can learn what parts they like or dislike, where they get stuck or confused, and what you can improve. 

Many UX research teams use in-house labs or office space to conduct in-person usability studies. But not all usability research needs to be conducted in person to be useful! It can also be done remotely.

What is remote usability testing?

Remote usability testing is a research methodology that uses an online software program to record the screen (and voice, depending on the tool you choose) of test participants as they use your site or app in their natural environment—at home, in their office, or even a specific location you ask them to go.

The benefit of this type of research is that your pool of potential study participants isn’t limited to a specific location. And you can conduct more research sessions in a shorter amount of time (and with less recruiting hassle). 

In the rest of this article, I’m going to share the benefits of remote usability testing, and I’m also going to break down the two different types of remote testing: moderated and unmoderated. I’ll dig into the pros and cons of both, and give you examples of when you should use them. Let’s dive right in. 

The benefits of remote usability testing

There are a few key benefits of conducting remote usability testing. First, research that happens in the participant’s natural environment can give you more realistic insights than lab research. 

Since participants aren’t in an artificial and unfamiliar setting, they’re more likely to give honest reactions and unbiased opinions. You also gather contextual findings of how your product is used naturally.

Plus, some research just can’t be done in a lab. 

If you’re trying to answer questions about how users interact with a product in a specific context (using a fitness app at the gym, for example), then you’ll need to make sure your research happens in that location. We call that destination-based testing. 

(Note: For research that isn’t dependent on a location, study participants can be in their homes, offices, or anywhere that is convenient.)

Also, there are times when you might need to observe something other than a digital experience on a computer or mobile device. 

For example, maybe you’re researching how customers find their way around a theme park, install a thermostat at home, or choose which brand of laundry detergent to buy at the store. These studies are called beyond-the-device tests, whereas traditional app and website studies are simply device tests.

Finally, it’s faster and easier to recruit test participants with online usability testing platforms that offer access to a panel of test participants. 

Rather than trying to find (and compensate) a group of target users on your own, you just designate your target audience and your testing platform will recruit users in their panel that match your description. This shortens the testing period to a few hours, rather than days or weeks, allowing you to get feedback and iterate right away.

Moderated vs. unmoderated testing

There are two different methods you can use when conducting a remote usability study: moderated or unmoderated testing. 

There are pros and cons to both methods, so how do you know which approach is right for you? Well, it’s going to depend on your specific situation and what your goals are. 

In the following sections, you’ll learn the difference between moderated and unmoderated remote usability testing, the pros and cons of each, and when you should use each of them. That way you can decide which approach makes the most sense for you. 

What is remote moderated usability testing?

In remote moderated testing, researchers (aka moderators) are in a different location than their test participants, but both parties are on a web or conference call and are sharing screens. Moderators can see what participants are doing, and they can both communicate with each other in real time.

The interaction between the moderator and target user is in real time. They are in the same “virtual” space and connect via a remote conferencing tools.

The moderator provides test participants with activities to complete while using a design or interface, and participants typically think out loud as they work on the tasks. The moderator observes the users as they work through the tasks, and the moderator may also ask questions for clarification or to gather more data.

These sessions should be centered around prompting the test participant to provide in-depth responses. Moderators will typically intervene only when necessary to help a test participant move through a difficult task or to probe for greater detail in an answer.

To illustrate what it looks like in action, the video below is an example of a moderated test on a very early-stage wireframe.

Sample moderated session

Pros of moderated usability testing

Moderated usability testing has two main advantages. First, moderators can interact with participants, meaning they can ask users to elaborate on their comments and ask them additional follow-up questions. 

For example, if during a study a user says, “This isn’t what I was expecting,” the moderator could ask the participant, “What were you expecting?” And the participant will likely elaborate based on that follow-up question. This is something that’s nearly impossible to do with unmoderated testing, and it’s especially useful for exploratory tests. 

The second advantage is the control and flexibility that comes with moderation. Moderators have control over the session and can corral the user is she gets off track or if the task isn’t worded clearly enough. 

You can also move tasks around or even eliminate tasks on the fly. Again, this isn’t really possible with unmoderated testing.

Cons of moderated usability testing

But the benefits of moderated testing mentioned above also present two specific disadvantages. Moderated testing requires a researcher to be present for the duration of all the sessions.

Since someone has to be there for every test, it takes more time. This can be a challenge, and it often results in a smaller sample size. Most moderated usability studies have between 5 – 15 test participants.

And the integrity of a moderated study depends on the skill of the moderator. An inexperienced moderator can actually influence the user’s behavior by giving them too much direction or asking leading questions—which can totally skew your data. 

There’s also some major logistical and technical drawbacks when conducting remote moderated testing. The sessions have to be set up with participants ahead of time. This requires a commitment from the participant and there’s a chance they will cancel or be a no-show. 

And because there’s an additional party that needs to connect to the session (the moderator), there’s also a higher chance that you’ll run into technical issues.

When to use moderated usability testing

There are specific situations when moderated testing is the best choice. We recommend using a moderator in these circumstances:

• If the test includes tasks that require hand-holding, probing, or in-depth follow-up questions based on user behavior and comments. A moderator can make sure participants don’t get stuck or off-track.

• If users have to complete a task successfully in order to move on to the next task. So if your tasks are dependent upon one another, then you might want to consider moderated testing. Do your best to avoid dependencies, but this isn’t always possible.

• If you can’t give users direct access to a design because of security or technical concerns around the site or product being tested, using a moderator to share her screen with the participant may be a better alternative.

What is remote unmoderated usability testing?

In a remote unmoderated usability test, researchers use a software program to automate their study. Test participants get a list of pre-determined tasks to perform on their own while their screen and voice are being recorded, and they typically think out loud as they complete the tasks.

The participant decides when and where she would like to complete the study, and uses an online tool to participate, provide feedback, and record the session.

They complete the test in their own environment without a moderator present. And when they’re done, the researcher receives the results in the form of a video recording (which they can review at a later time). 

In an unmoderated test, the researcher doesn’t communicate with the participant in real-time. But the researcher may include follow-up questions as part of the study, or they may follow up with the participant after viewing the session.

Below is an example of what a remote unmoderated test looks like in practice.

Sample unmoderated session

Pros of unmoderated usability testing

There are a lot of advantages to unmoderated usability testing. First of all, a researcher doesn’t need to be present for the duration of all the sessions. And that means multiple sessions can be completed concurrently. This allows you to use larger sample sizes, which reduces the margin of error and often allows for more significant data. 

Also, standard in-person test usually takes about three weeks to plan the study and find the users for the session. And typically another three to five days for travel prep and conducting the actual session. 

With remote unmoderated usability testing, you can usually get results back within a day or two. And sometimes within hours (especially if the tool you choose automates the recruiting process for you).

Finally, participants can complete the study when it’s convenient for them, so there’s no need to schedule or set up appointments. 

Cons of unmoderated usability testing

The qualities that make unmoderated testing useful also bring their own unique challenges. 

Since a moderator isn’t present, you can’t interact with participants while they work on tasks. There’s no way to probe or ask follow-up questions based on their comments or activities in real time, so you have to write a test plan that’s easy for your participants to follow.

And you also have to predict your follow-up questions and hope they’re relevant to users at the end of the session. On top of that, you can’t guide them back to the task if they get sidetracked or if they don’t follow the task directions.

And even though you don’t need a moderator present for all of the sessions, you still need a researcher to watch the video sessions, collect data, and gather insights from the recordings (although some platforms will review the videos for you if you don’t have the time). 

When to use unmoderated usability testing

There are specific situations when unmoderated testing is the ideal testing method. We recommend using an unmoderated usability testing software platform under these circumstances:

• If there are very specific or direct tasks, especially those focused on interaction design (for example, asking a user to find the price of an item, or where a company is located). These typically don’t include a ton of exploring and the chance of the user getting off-track is slim.

• If you need a larger sample size, then unmoderated testing is more appropriate since a moderator doesn’t need to be present during each session.

• If you want to capture natural use, especially over time. When users are asked to record themselves every time they use a site or product, there’s no real need for moderation.

• If you want to gather feedback quickly. Unless you are looking for a very narrow demographic, a study can often be completed in one day! If you don’t have a specific need to do a moderated session, unmoderated is the way to go because it’s faster and easier!

The final word

Traditional usability testing has been conducted in-lab, but remote usability testing allows you to test more participants quicker for a much lower cost. It’s like a secret weapon for teams who iteratively test and implement feedback.

It’s also important to remember that every user research methodology has distinct advantages and disadvantages. And understanding the unique qualities and limitations of each approach can help you choose which one is right for you.

If you want to learn all of the various types of research methodologies, when to use them, and how to organize and share your results, check out The UX Research Methodology Guidebook. It’s for anyone who’s ready to get started testing, but would like a little guidance on how to approach a study, when to run it, and how to interpret the results.