Usability testing is a means for measuring how well people can actually use something (such as a web page, a computer interface, a document, or a device) for its intended purpose. If users, or test subjects, have difficulty understanding instructions, manipulating parts, or interpreting feedback, then the developers must go back to the drawing board, improve the design, and test it again. During usability testing, developers are not expected to explain their product to the user, or argue about its merits. The aim is for them to observe a real user use their product in as realistic a situation as possible, so as to discover errors and possible areas of improvement. A common mistake that designers make, for instance, is to focus too much on creating designs that look "cool", but compromise on usability and functionality.
- "Caution: simply gathering opinions is not usability testing -- you must arrange an experiment that measures a subject's ability to use your document." 1
Setting up a usability test involves carefully creating a scenario, or realistic situation, wherein the user can perform a list of tasks using the product being tested while observers watch and take notes. Several other test instruments such as scripted instructions, paper prototypes, and pre- and post-test questionnaires are also used to gather user feedback on the product being tested. For example, if the aim is to test the attachment function of an e-mail program, the scenario would describe a situation where the user needs to send an e-mail attachment, and ask him or her to go through all the steps to perform this task. The aim is to observe users function in a realistic setting, performing realistic tasks, so that developers can see where they face problems, and what they like. The technique popularly used to gather data during a usability test is called a talk aloud protocol.
What to Measure.
Usability testing generally involves measuring how well test subjects respond in four areas: time on task, accuracy, recall, and emotional response. The results of the first test are the baseline or control measurement; all subsequent tests are compared to the baseline.
- Time on Task -- How long does it take users to complete a set of basic tasks? (For example, find something you want to buy, create a new user account, and order the item.)
- Accuracy -- How many mistakes did users make? (Can the user correct these errors, if given the proper feedback, or are the errors fatal?)
- Recall -- How much information does the user remember, after completing the assigned tasks?
- Emotional Response -- How does the user feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)
Bruce Tognazzini advocates "close-coupled testing": "Run a test subject through the product, figure out what's wrong, change it, and repeat until everything works. Using this technique, I've gone through seven design iterations in three-and-a-half days, testing in the morning, changing the prototype at noon, testing in the afternoon, and making more elaborate changes at night." 3
See also:
Usability Resources Online:
- Usability.gov
- Jakob Nielsen: UseIT.com
- Dennis G. Jerz: Why Usability Testing Matters: Palm Beach County Ballot Design
- Bruce Tognazzini: How to Deliver a (Usability) Report Without Getting Lynched
- John S. Rhodes: WebWord: Usability and Human Factors on the Internet
- Usableweb
- Lyle Kantrovich: Croc O' Lyle