The System Usability Scale as a tool of usability design

How can we measure usability? This is what Atostek user experience design specialist Markus Jaakola has to ponder.

Usability is a feature that is required of software, yet that is challenging to measure objectively. When measuring something, it must always be compared to something else. But there is no absolute scale or metric of usability that every system could be compared to.

Defining and measuring usability

Traditionally, usability has been defined as referring to system functionality for its intended purpose. The traditional definition of usability is based on three factors:

• effectiveness of use
• efficiency of use
• user satisfaction.

The metrics of effectiveness, efficiency and satisfaction can be defined system by system. Each software meets some usability requirements set for it. However, software are different from each other in both purpose and function, so the usability requirements of software are also different.

When we say software differ, we mean that they have different intended uses, different contexts of use and different users. Of course, the implementation technology and user interface can also be anything in the world.

When usability requirements set for the system are met, its usability can be said to be at a good level. But how good is it compared to other systems? How can we compare the usability of different systems? This is the question John Brooke was trying to answer when he developed the System Usability Scale in 1986.

The SUS survey method

The System Usability Scale, or SUS, is a simple and reliable tool for measuring the usability of any system. These systems may include software, hardware or a combination of the two.

The SUS method consists of a ten-point questionnaire that users take after using a system or its prototype. The SUS survey questions are multiple-choice questions with a five-level Likert scale, so answering is quick and easy. The result of the survey is a score describing the usability of the system on a scale from zero to one hundred.

The score obtained from the SUS calculation describes system usability on a general level. The SUS survey does not include detailed feedback, so you cannot tell what needs to be improved based on it alone.

The method is based on several hundred system ratings, so we know the average result produced by the survey, regardless of system. Therefore with the results, the target system is compared to all other existing systems.

The strengths of this method are simplicity and reliability. You can get a reliable result with a very small number of respondents. Even just a handful of respondents are enough to give a result providing a correct idea of the real state of the system.

The lightweight survey is easy to implement both on paper and online. No special software is required to analyze the results. You can calculate the result with the formula openly provided by the method. Atostek’s UX experts have developed ready-made survey and formula templates that make usability measurement easy and cost-effective.

Challenges of the SUS method

One of the challenges of the method is interpretation of the resulting score. Although the scale runs from 0 to 100, the average result is not halfway through the scale, but at 68 points. This is unintuitive, so when reporting on the results, it is better to use percentiles describing the result distributions. Percentiles describe how many of the data points fall below the value in question.

For example, 68 is the average score, so 50% of all SUS results fall below 68 points, and correspondingly, the other 50% of results fall above it. If the score is 80 points, that is an excellent result belonging to the 90th percentile. With a score like that, we can say that 90% of all systems are worse off in terms of usability than the system in question.

Another challenge is the quantitative and abstract nature of the results. The method provides information on the level of usability but not the reasons behind it. The method does not make respondents give reasons for their answers, instead focusing on getting results quickly and easily.

Usability experts must use other methods in tandem with the SUS in order to identify what can be improved and what needs to change about the system.

Benefits and using the scale

The System Usability Scale is just one tool for measuring usability. It alone is not enough for designing and evaluating usability, but it enables us to compare different systems with an universal metric.

The SUS method can be used for software development to, for example, verify the fulfillment of usability goals or assist in choosing the solution with the best usability from several systems.

If you are replacing an old data system with a new one, the goal is usually to improve usability at the same time. With the SUS method, the fulfillment of the goal can be measured and verified numerically in a scientifically valid way. This way, success can be measured by more than just how you happen to feel about it.

Atostek’s toolkit of service and UX design services brings you a variety of tools for a successful software project. One of them is the System Usability Scale presented here.

Sources:

Brooke, John. (1995). SUS: A quick and dirty usability scale. Usability Eval. Ind.. 189.

Nielsen Norman Group: Beyond the NPS: Measuring Perceived Usability with the SUS, NASA-TLX, and the Single Ease Question After Tasks and Usability Tests. https://www.nngroup.com/articles/measuring-perceived-usability (Cited May 19, 2021)

Usability.gov: System Usability Scale (SUS). https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html (Cited May 19, 2021)

---