ash dzick Get In Touch

Get In Touch

Prefer using email? Say hi at a.dzick@gmail.com

Go Home or Go Remote: Usability Testing Anywhere

You know what you want to test. You know what you want to ask during your test. Now all you need are some participants. Usability testing can be done with anyone and anywhere. Traditionally, usability tests are conducted in person, either by yourself or by a recruiting firm. As a user experience team grows, they also start to reach out to more clients through remote moderated testing. In the last few years, websites such as UserZoom and UserTesting.com have enabled teams of all sizes to be able to test with a wide variety of different subjects through remote unmoderated testing.

In-person, remote moderated and remote unmoderated all offer different access to participants, and each carries distinct advantages.


In-Person

Usability testing in person is a great way to start off. Testers don’t have to worry about setting up cameras, conference calls, etc. It’s easier to measure behavior and non-verbals when you’re observing in person. Participants get to interact with a human in front of them, making it easier to start a dialog. You might end up off-script more, but the insights are just as valuable.

There’s also a lot of options for in-person testing. It can be done for little to no money. Starting off, all you really need is a way to take notes. Recording to review later is a nice extra, but not required. A number of testing suites, like Morae and Silverback, can elevate your in-person testing with more analytics, analysis, and notes organization.

Benefits

  • In person is easier to start with and fewer things can go wrong.
  • In person can help you better measure behaviors and non-verbal reactions, instead of relying on opinions and talking out loud.
  • When you’re visiting customers in person they feel valued, helping you to build a strong customer relationship.

Drawbacks

  • When you focus only on one area, it is harder to specialize your recruitment efforts and you may end up with participants with location bias.
  • If you’re doing all your own testing, especially if you designed it, you can accidentally introduce your own cognitive biases.
  • If you don’t have a testing space available to you, you might have to rent one.

Remote Moderated

Remote moderated testing is conducted by yourself or on your behalf by a testing firm. Remote moderated testing still has someone watching the test and interacting with the user, they just aren’t in the same location as the user. It adds an additional element of phone or video conferencing that needs to be counted for. Screen sharing still enables tests to be able to easily follow up questions and guide the user.

Many researchers find it easier to be more impartial during remote testing because they don’t have the emotional element of interacting with a human face-to-face. Remote testing does make it harder to measure non-verbal communication, which is an important part of gauging reactions and building trust.

Benefits

  • Remote moderated testing enables researchers to research a large and more diverse set of users.
  • Testing through remote methods decreases the chance of location bias and the researcher’s own cognitive biases.
  • When done in-house and with your own user recruitment, moderated remote testing can be done very inexpensively.

Drawbacks

  • You lose the humanistic approach of talking face-to-face with users.
  • Anything with complex security or sensitive information is significantly harder when testing remotely.
  • Anything with physical products, user movement or eye-tracking as part of the usability testing will suffer from being remote.

Remote Unmoderated

Unmoderated testing adds an additional element to remote testing. Instead of personally administering a usability test, the test is wholly handed off for a user to complete on their own. The user completes a series of tasks, moving onto the next one when they have either completed or given up on a task. As part of testing, users will consent to record their screen and talk aloud. Metrics usually include heat maps, user clicks, and self-reported completion data or opinions. Unmoderated testing does not have any observation of the user, so you do not gain any nonverbal communication observation.

Common options for unmoderated testing include UserZoom, UserTesting.com, ZURB, Loop11, and UsabilityHub.

Benefits

  • It’s easier to recruit a specific audience, making it very easy to get a very targeted or diverse user base.
  • Many of the unmoderated testing suites offer a free trial, making the barrier to entry very low.
  • You can free up your researcher’s time to work on other testing that might have a more significant impact or higher priority.

Drawbacks

  • It can get expensive very quickly.
  • You can get a lot more information than you do with in-person testing, but to find the valuable pieces you have to sift through it all.
  • It’s self-guided with instructions, so you’re relying on the user to drive.
  • Some testing suites have users who take tests regularly, becoming “testing pros” and can influence the final result.

Which should you choose?

If you’re entirely new to testing, I recommend testing in person. Spend the time face-to-face to build up your confidence in directing users to complete tasks. Work on your ability to measure non-verbals and start getting other people involved.

For testing a smaller workflow or when diversity in user base isn’t as important, either remote moderated or in-person testing is fine. Choose whichever gives you quick results and work iteratively.

Designs that are targeting a particular audience or need a lot of test participants can be tested through remote moderated or remote unmoderated testing. Remote gives a broader audience base and enables you to reach out to more current and potential users.

Whether you choose one method or a combination, the important thing is to get out and start testing your design decisions.