When to use each type of usability testing?

Published on January 14th 2020.  

In this chapter, we look at the role usability testing plays in Design Thinking We also discuss when to use what type of usability testing, but keep in mind that the general rule is any type of testing is better than no testing at all. 

However, depending on what stage of the development cycle you are in, certain types of usability testing are better suited.  

In other articles, we look at what usability testing is, the number of users you need for usability testing and how you can analyze usability test results.

The general rule is that you are always better of conducting even a few usability tests than no tests at all. Read more usability testing in the article on usability testing. 

Below we discuss the role of usability testing in the following scenarios:

1. You have a website or app that is already live, but you're curious how you can improve the UX
2. You are developing a website or app and want to test a prototype
   1. Testing usability for mock-ups and non-interactive prototypes
   2. Testing prototypes with interactive elements
   3. Testing the usability in the phase prior to launch

The advantage of having an launched your website or app is that you are probably working with Google Analytics and Hotjar to drive optimizations. 

Great choice. They are amazing tools (we use them as well), but it is still quite difficult to find out what exactly is not workring. For example, GA and Hotjar can show you that users often drop out at a certain point, but it's still guesswork trying to decide why they drop out.

With usability testing you can eliminate the guesswork, as testers think out loud and provide you with valuable feedback whilst doing so. 

In short, you can find out what users do with Google Analtyics and Hotjar and you find out why they do it with usability testing. 

Sounds great - but how frequently should I conduct usability tests? 
 

In the ideal world, usability testing is part of the optimization process and user testing takes place every month. In this way, continuous optimizations and testing takes place.

Large companies often have teams that are solely concerned with optimizing the user experience (UX), making it easier to structurally deploy usability testing.

Of course, this might not be feasible if you work within a smaller organization. In that case, it works well to setup a fixed number of usability tests to be conducted every two to three months. By doing so, you have enough time to implement and test new features when working with a smaller budget. 

Curious what we recommend for your organization? Reach out to us.

This type of test is also called an exploratory user test. Sketches and non-interactive prototypes are tested with real users. Since users are unable to interact with these early designs, it is recommended to have a moderator guiding the session. 

These types of usability tests are usually conducted in a usability lab, but there are tools (e.g. Invision & Adobe XD) out there to help you do this faster and quicker online. 

In this early phase of the development process, design teams are usually interested to find out to what extent the design:

• Helps the user the achieve fulfill its needs;
• Clarifies which steps the user must complete;
• Makes navigating to different pages easy and intuitive.

Since fundamental design decisions often take place early in the design process, the value of conducting usability tests early cannot be understated. If radical adjustments in the design have to be made, it is better to do this early in the design process - as doing this in a later stage will require a lot of time and effort.

This type of test is also called the assessment test. In this phase you test a product that has already built-in some functionalities and in which the tester can move through the website or app. Depending on the complexity of the design and the process, this can be done with or without a moderator. 

This type of test can take place in a usability lab or via a usability testing platform like User Sense. 

Key characteristics of assessment tests are: 

• Instead of just verbal feedback, you actually see how users interact with the design
• Communication with a moderator is restricted or absent
• Quantitative metrics such as the error rate and the task completion time are collected

This type of test is also called the validation or verification test and takes place at the end of the development cycle. This type of usability test can take place online and in a usability lab, with or without a moderator. 

Key characteristics of the validation test are:

• Usability criteria that have been setup prior to the test and will be measured during the usability tests
• The tester has very limited or no contact at all with the moderator
• Collecting quantitative data has a bigger focus compared to other tests

Once the above data points have been collected, they are then compared to a project, company, or industry benchmark to gain insight into how the product is performing. 

Usability metrics are discussed in the chapter on ‘Analysing Usability Test Results’. 

This type of test is also called the comparative user test and can be used during every phase of the design cycle. In a comparative test, multiple designs are tested. Companies can do this by creating multiple designs themselves, but also by comparing their own designs with that of the competition.

By having multiple variants tested, you gain insight into what works and what doesn't. The goal of this type of usability testing is not to see which design is better, but to pick and choose elements of all designs that worked well - and then combine these into a new design.

Design thinking is a fairly new but prominent methodology for developing new products or services. Unlike other development processes, with Design Thinking the users’ needs and wants are at the core of all decisions. In order to identify these needs, wants and pains a lot of time and effort is put into user research prior to development. 

In order to develop user-friendly and usable products, it is crucial to test prototypes, websites and apps with real users. When it comes to usability testing within the Design Thinking Process, there are two things to keep in mind:

• Testing in the development cycle pays off. Early testing in the development cycle shows how users handle the prototype and provide insight into what works and what doesn't. Modifications to the design can be taken into account immediately, without it costing much time or money.

• Design is never finished. Even when a website or app has been launched, the (structural) implementation of usability testing pays off. This gives companies insight into the rapidly changing needs of users and helps to maintain a competitive advantage.

There is much more to write about Design Thinking and it’s benefits, but that is beyond the scope of this article. For more information about Design Thinking, we recommend this article.