What is usability?

Usability is often defined as “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.”

This definition is provided by the International Organization for Standardization (ISO), within the standards 9241-11 (2018) and 9241-210 (2019). However, there are many other definitions of usability to be found - we decided to stick with ISO. The usability methods and measures provided by the Interactive Usability Toolbox are categorized by the framework of ISO TR 16982 (2002). This framework describes method types, their application potential and fittings for different evaluation contexts.

Functionality, ease of use, comfort
- how about that?

It can be hard to wrap your head around the term usability. We understand that users and developers like to describe their usage experience with specific usability attributes.

But how is an attribute like "ease of use" defined? To what extent does it describe a system's effectiveness, efficiency, and satisfaction? Our toolbox supports 49 different attributes which were carefully studied and categorized. Learn more about each in our Glossary.

Context matters!

The usability of a system is always defined by the context of use. Usage experiences can strongly differ when testing the same design with another user, varying environments, or following different tasks.

Understanding and specifying the context of use is an essential step of user-centered design and usability evaluation. Can you assume that your device will work in dynamic, real-life situations if you only tested it in the controlled lab environment?

Which methods make sense?

The complex human-robot interaction that users of wearable robotics encounter have many facets which can and should be evaluated.

The usability methods and measures provided on our toolbox are categorized by the framework of ISO TR 16982 (2002), which describes method types, their application potential and fittings for different evaluation contexts. Each method has their own pros and cons. You decide which to use based on the insights you would like to get.

Who are your target users?

Understanding, evaluating, and integrating human factors is a critical challenge in the design of wearable robotics. Your context of use starts with the user, not the technology.

Who are the primary users? Are there secondary user groups? Make sure you understand your target users and involve them in the design early on. Valuable insights can be generated by evaluating your solutions with secondary user groups, like therapists, caregivers, or family members.

Define your scope

How many participants do you need for your usability study? This mostly depends on your research intentions and resources.

One can value a single user’s opinion to trigger design changes, but it is also essential to distinguish between personal preferences and global usability issues. According to Nielsen and Landauer (1993), a group of five users will be able to identify 80% of your device’s usability issues. However, your study will likely require more than 20 subjects to prove your device intervention effectiveness statistically.

Find the right balance

We recommend to consciously distribute evaluation efforts between the usability dimensions effectiveness, efficiency, and satisfaction.

While this distribution will likely adapt as technology matures it is important to consider efficiency and satisfaction from the very start of a WRD project. This implies that developers should be inclined to include target users as soon as possible, also the early stages of WRD development. A simple walk-through, interviews, or focus groups with target users can shape your concept or low-fidelity prototype in the right direction from the start.

Learn to value qualitative data

In contrast to what is mainly reported in scientific literature, a usability evaluation protocol should include substantial qualitative measures.

While quantitative data allows for comparison to previous design iterations and the state of the art, qualitative data helps identifying usability issues that may not be obvious to WRD developers when only collecting and comparing numbers. Also, qualitative evaluation by nature calls for more personal interaction with the target users and their satisfaction with the device. There are two main standards that can be used to ensure the quality of the data collected with qualitative research: COREQ and SRQR.

Seek for comparability

The wearable robotics field is missing evaluation benchmarks. For benchmarks to arise, we need to start using similar, validated items that generate comparable outcomes.

How about using a usability evaluation protocol with 2/3 of reproducible, standardized measures and 1/3 of customized tools? This will likely increase data validity and generalizability of your data and could help to establish usability evaluation benchmarks.

It is never too early

Before any prototype is built, assess whether there is a real need and interest by end users to develop a technology. Reach users, ask them and observe which are their real needs. When you identify the one you want to address, validate with the users if they are interested in solving it with a WRD. Is it really the best solution? Only if the advantages of a WRD are clear above other solutions and users show interest in a WRD as a solution, move on to design your device.

This early need elicitation and validation can be easily performed with methodologies like contextual interviews, user Interviews, focus groups, user observation or task analysis.

Iterative evaluation

Implementing UCD iteratively throughout the design and development of a device from beginning to end will ensure that your device’s components and features work well for the intended users and context, and thus your device’s usability. It is a way to avoid problems due to overlooking important user requirements after a device has been fully developed, that might undermine its pathway towards reaching the market and end users.

In other words, implementing UCD from the first stages of technology design saves time, money, and effort!

What about ethics?

Although the methods to implement UCD are not invasive or risky, it is always necessary to have approval from an ethics committee before reaching the users, especially if people with any kind of impairment, illness, or medical condition are involved.

When implementing UCD, researchers are treating people’s personal and sometimes medical data, as well as their perspectives, beliefs and habits. Thus, a protocol approved by an ethics committee guarantees the protection and appropriate management of such data. Users must be informed of the goal of the study and provide informed consent before participating.

The vision

If you shouldn't compare apples and oranges, why should you do the same with usability protocols?

The Interactive Usability Toolbox is designed to help you define your context of use and find measures that fit it. Furthermore, we can tell you if other developers have used those measures in comparable studies. This way, you will use similar protocols like others, which allows you to compare your results and insights.

The backbone

Each item in our usability evaluation database (150+ methods/measures) has been carefully studied to characterize the context fit to wearable robotics applications. Some tools only apply to individuals with spinal cord injuries, others are specifically validated with children. Furthermore, the evaluation scope of each measure is described using our extensive list of usability attributes.

Use our Wizard to find all measures fitting your context of use and evaluation scope!

Based on real world data

Our database is continuously growing and improving. The data comes from literature reviews, individual studies, websites, own experiences, and dedicated surveys. Currently, our database contains 142 usability studies shared by 35 contributors around the world.

We constantly learn about more tools that can be used for wearable robotics usability evaluation. With the recorded use function, we are able to show you if and to what extent specific tools have been used for the same context of use before.

Become an active contributor

Our aim is to make the database, and our recommendations as representative of current practices as possible. To do that, we need more real-world data - from you!

We welcome you to sign up as a contributor and start helping us to reach our shared vision of promoting user-centered design.

Tell us about your usability evaluations

As a contributor, you can submit requests to add data to our database. This will help expand the database, and improve the 'Recorded Use' ratings of each existing evaluation item. With this, we will be able to make better, more context-specific recommendations.

Tell us which usability evaluation methods you used in the development of your wearable robot and try to be as specific as possible when explaining your context of use.

Help us to continuously improve

Let's improve this platform and its content together. We can use your help with constructive feedback and specific improvement suggestions. Let us know your usability evaluation experiences, suggestions, or if you have a great idea worth exploring by submitting open feedback in the contribution page. Thank you for your help!

Websites

Rehabilitation measures database:
• https://www.sralab.org/rehabilitation-measures
Framework for bipedal robots:
• https://eurobench2020.eu/
Platform to connect with participants:
• https://www.userinterviews.com
Consulting firm for user experience:
• https://www.nngroup.com/
General usability concepts and tools:
• https://www.usability.gov
• https://www.usabilitybok.org
• https://www.usabilityfirst.com/

Recommended books

• Norman, Don. The design of everyday things: Revised and expanded edition. Basic books, 2013.
• Ku, Bon, and Ellen Lupton. Health Design Thinking: Creating Products and Services for Better Health. MIT Press, 2020.
• Rubin, Jeffrey, and Dana Chisnell. Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley & Sons, 2008.
• Albert, William, and Thomas Tullis. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Newnes, 2013