Using SUS scores to benchmark the usability of your digital products

The System Usability Scale (SUS) is a quick and easy way to evaluate the usability of a product. It was created by John Brooke in 1986 and is often referred to as a "quick and dirty usability scale".

The SUS is used to measure things such as the perceived ease of use and is employed across various digital products and services to help UX practitioners determine if there is an overall problem with a design solution. However, it’s important to note that the SUS is not a diagnostic tool, but rather a means of providing an overall assessment measurement.

The SUS is made up of 3 key characteristics:

• Effectiveness - whether users are able to successfully achieve their objectives

• Efficiency - refers to how much effort and resource is used in order to achieve those objectives

• Satisfaction - an understanding of whether the experience was satisfactory for the user

Users are presented with a 10-question SUS questionnaire after a usability testing session.

The aim of the questionnaire is to be able to get quick and unfiltered feedback from the user in an unmoderated environment. When carrying this out, you need to make sure that the project has been described well to avoid any confusion with users.

When using this questionnaire to gain feedback, we always aim to carry out this process with a minimum of 5 users to ensure reliability in your results. If you can speak to a larger sample size, this is even better to obtain more representative results. You should allow 1 – 2 minutes for users to complete the questionnaire, with only ranking scores being collected as feedback.

Below are the 10 System Usability Scale questions. Statements are answered using a 5-point Likert scale — participants rank each question from 1 – 5 based on how much they agree with the statements, with 5 being complete agreement.

1. I think that I would like to use this [project] frequently.

2. I found the [project] unnecessarily complex.

3. I thought the [project] was easy to use.

4. I think that I would need the support of a technical person to be able to use this [project].

5. I found the various functions in this [project] were well integrated.

6. I thought there was too much inconsistency in this [project].

7. I imagine that most people would learn to use this [project] very quickly.

8. I found the [project] very cumbersome to use.

9. I felt very confident using the [project].

10. I needed to learn a lot of things before I could get going with this [project].

To generate the final SUS scores, your UX researcher will follow a predetermined scoring tabulation methodology to get a SUS score out of 100.

The average SUS score is 68, so scoring above or below this figure will provide some immediate insight into the usability of your digital product. A score below 50 indicates a particularly poor user experience, and anything above an 85 would be considered excellent with minimal need for improvements.

The SUS scoring system can sometimes be seen as complicated. As results come out on a scale of 0 – 100, they are often interpreted as a percentage. This is not the case — they are percentiles, therefore it’s necessary to be aware of this when clearly communicating these findings with stakeholders.

SUS scores provide a quick and reliable method of assessing the usability of your digital design solutions. They have a variety of use cases in the context of digital product design in pharma and healthcare, such as:

• Test the success of design changes: by comparing SUS scores taken before and after design changes have been implemented across your product or service, you can tangibly demonstrate the positive impact to product stakeholders — e.g. Average SUS scores improved by 35% in 6 months due to implementing UX or UI design changes.

See a real-life example from our work with healthcare provider, Vhi, here.

• A/B testing between prototype designs: by giving half of your users one design and half another, you can clearly compare how users respond to different design concepts in order to make informed decisions when finalising designs and moving forward into development.
  
  

• Evaluate the usability of prototypes: as SUS scores are relatively quick and low effort to obtain, they can be used regularly to evaluate the usability of your digital product and keep you on track — for example, at the end of each design sprint. They can help uncover any issues so that they can be resolved early and avoid having to backtrack or waste effort and resource on developing solutions that your customers find difficult to use.

• Easy to create and scale: SUS scoring processes are quick and easy to create and build into your design process as there is a ready-made format to follow in regards to questions, but also how to analyse and interpret results. They can be carried out with small or large sample groups of users easily.

• Valid: SUS scores allow you to clearly differentiate between usable and unusable digital solutions, ascertaining the validity of your product design decisions.

• Reliable: Using SUS scores has been shown to be more reliable than custom questionnaires made case-by-case by researchers and product teams. They are able to detect differences and come to conclusions using a smaller sample size.

• Affordable and fast: The short set of questions means that it will not require too much time and effort for your UX researcher to administer SUS testing, which is particularly useful if you’re on a tight budget.