Assessing Usability 4: User Testing
Usually when you hear the term "usability testing" it is associated with some form of the techniques described below. As with some of the inspection techniques described in Part 3 of this series, formal user testing requires a pre-defined set of example user tasks that need to be completed. In this case, however, actual users are employed (participants are typically paid for their time) to complete the identified tasks.
Formal usability tests are typically well-controlled research experiments wherein actual end-users are asked to perform specific tasks with the product (or product prototype) while data is collected. Video and voice recordings are usually collected along with error data, action counts, success rates, time-on-task measures, and many other types of both cognitive (when available) and physical performance metrics. The data is analyzed offline and results are usually presented in a research report accompanied by recommendations for fixing any of the problems discovered.
Unfortunately, conducting these types of tests can be somewhat costly, as the video and voice recording equipment, the recommended dedicated lab space, and the recruiting and screening of qualified participants are all key to testing success. Despite these costs, however, the benefit gained from conducting a formal usability test, especially when conducted with a prototype, will be invaluable to your team.
Note that participating users should not have been involved with any of the product design process and should be representative of the intended audiences for whom the product is being designed.
"Paper" Prototype Testing
The first thing you should realize is that "paper" prototypes do not necessarily have to be made of paper. In fact, you can create a prototype of your product using just about anything, so long as your users have enough imagination to get over the fact that your "website" may actually be drawn on a chalkboard, or your new TV remote control is a cardboard box with buttons taped on the front.
The key to successful testing with a paper prototype is to present the user with a physical, yet abstracted, conceptual version of the final product, with instructions on how the product works in its current form. For example, user-interface prototypes created using a whiteboard and sticky notes or sketches in a notebook should be accompanied by detailed descriptions on what would happen if the user were to "press" a certain control on the "screen."
These types of low-fidelity prototypes may at first seem unprofessional (and even silly) to users - something which you may be able to use to your advantage. Many user testing labs employ a large, one-way mirror to allow test experts to inconspicuously observe a user's behavior. However, to the user, there's nothing inconspicuous about being placed in a room with a single large mirror directly in front of the table at which they're being asked to sit. Many people will feel nervous and self-conscious in this type of setting, so encourage them to have a little fun with the testing.
You may even find that users want to offer some criticism of the prototype as a way to "level the playing field" a bit - to put you on the spot for a change. Let them. The feedback they offer can be used to help your team develop better prototypes!
Medium-Fidelity Prototype Testing
Any time you're testing with a prototype, it's pretty obvious to your end users that they're not using the final product - there's still the odd "i" that needs dotting and a "t" or two that could stand to be crossed. While medium-fidelity prototypes still have a few rough edges, they are of a much higher fidelity that the "paper" prototypes used above. As such, they are a better approximation of the final product and, in testing, will typically generate results that are more applicable to the real-world situation
This increased realism does come at a cost, however. For example, if you are creating an e-commerce web site, a limited-functionality prototype will not have a dynamic shopping cart, but it will be created in HTML, be presented to the user in a web browser and have static screens representing the check-out process. Or, if your prototype is the TV remote control introduced above, it will typically be a wood, foam, or plastic model with buttons that have the form, location, and activation needed to properly evaluate usability during testing, even though they may not doing anything when pressed.
Testing with this type of prototype is qualitatively similar to the type of testing you would conduct with paper prototypes, although your capability to collect data can sometimes be increased, depending on the tools being used. (Web servers, for example, typically keep time-stamped access logs to record page hits.)
Depending on your development environment, you may also find that some of the HTML code used for the prototype can be re-purposed for the final product, cutting costs in the final development stage. However, the prototype should not be developed with "final use" in mind. As costly as it may be to do so, you should consider the prototype a throw-away part of the overall usability assessment process. Focus more on the data you collect, what it means, and how to fix problems you encounter and the ROI will more than make up for the costs incurred.
Full System Testing
If it were possible, we would completely prohibit the marketing of any product on which the only usability test conducted was using the fully developed, complete pre-launch system. Once a product is essentially finished, it's pointless to try to actively discover any usability problems - they are just too costly, if not impossible, to fix at this point.
And, by the time the next version of the product is ready for design, fixing the problems found during the previous test are typically considered to be less important than adding the latest must-have features and functionality (and revenue!) to meet market demands. Because of this, many times the usability test conducted on the full system at the end of product development is simply an exercise in "rubber stamping" the final product. Avoid this temptation.
Instead, use the time you'd otherwise spend patting yourself on the back for a job well done (or, more likely, sugar-coating product deficiencies in official company statements) developing a comprehensive plan to inject the usability evaluations and assessments described in these articles into the next product release. Start this next product development cycle with a critical look at usability using the actual product you've just launched.
There's a much better chance that you'll actually use the data from the usability test if you collect it only when you're actually ready to deal with it. The results may still not be pretty (time doesn't heal all wounds) but at least you'll be able to properly address the issues by scheduling time for various usability assessments into your project plans, timelines, and other project management deliverables, instead of just trying to force-fit them in after the fact.
_______________________________
Note that as we've discussed usability assessment techniques in these articles, we've moved closer and closer to evaluating the use of the final product in the real-world setting. The reason why we've emphasized the use of different types of assessment techniques earlier in the design process is due to the fact that the cost associated with making a change increases the farther along you get in the product development lifecycle.
However, if you have consistently conducted usability assessments while the project progressed, you really shouldn't uncover any "major" usability issues while using the user testing methods described in this last installment - which is a really good thing. Trust us.