I recently worked on an application where, when users first saw it, they couldn’t find the “Add New Item” button. That was because, to add a new item, users first had to do a search of all the items listed in the database, select one item (any one item at all) to view, and view that item to (finally) get an ”Add New Item”. Once the issue was pointed out, it was such an obviously stupid design that we wondered how we’d missed it during development.
This isn’t, however, an unusual case — there are so many things that can go wrong with a user interface that there is no possible way that any team will get the UI right the first time. The team that builds the application will, unfortunately, be the group least likely to notice these problems because they know far too much about the application (the “No one has ugly children” rule applies here). Only usability testing can find these problems so that the team can fix them.
What I’m saying is that the biggest danger associated with usability testing is not that it will be done badly but that it won’t be done at all. The two most common reason teams give for skipping usability testing are, “We don’t have the time” and “Our testing never finds any problems (probably because we don’t have a lab).”
I can’t do much about the “We don’t have time argument” except to say that if you don’t take the time, you’ll just waste the time of everyone who uses your application until you finally take the time to fix your UI. A usability test requires a team member, a user, an hour of preparation time, a half-hour to conduct…and as much time as required to address the problems the test unearths. But the test itself is not a time consuming activity.
As for the second reason: There are two symptoms that flag that a team is doing usability testing poorly. Immediately after the test, you hear, “No one had any real problems.” Two or three months after implementation, when the UI is finally fixed (or the users have adapted to the UI), you hear “Well, those problems didn’t turn up in our usability testing.” These teams often come to the conclusion that the reason their testing didn’t turn up any problems is because they didn’t have a usability lab and, since they can’t get a usability lab, they should give up on what little usability testing they’re doing. In fact, as these teams demonstrate, it’s very easy to do usability testing in a way that suppresses problems.
When teams say that their usability testing didn’t find any problems it means that the testing was done poorly, not that they didn’t have a lab. In fact, having a usability lab is often the reason that usability testing doesn’t find any problems (and there are always problems).
As an example, I showed up at one client’s site to do some usability testing and was shown into a board room that I had never seen before. It was a lovely room and was fully equipped with everything we would need for our testing. Still, I asked why we were here. “Well,” I was told, “The shop floor where the application is going to be used is really noisy and distracting so we figured you’d want someplace quieter to do your testing.” I explained that testing the UI in an environment that was so different from where the application was going to be used wouldn’t actually tell us anything useful and my client relocated us to an environment more like the shop floor.
First rule: Test where the application will be used (or somewhere like it). This is especially true with mobile design.
While developers know too much about how the application works to expose any problems with the UI, most of the people in the world don’t know enough about the application to be able to test it. User interfaces are designed to enable a particular group of people to perform a particular set of tasks. Having your application tested by anyone else is a waste of time for all of the people involved.
Second rule: Have your UI tested by the people who will actually use it, doing tasks they will actually perform with it.
Another reason that usability testing fails is that users don’t tell the testers about the problems they find. It turns out to be very easy to convince users that they are what is being tested, not the UI. As a result, when the UI fails, the users will blame themselves and tell the team, “No, it’s alright. We’ll know how to do that.”
Third rule: Emphasize to the user that they are the tester, not the “thing being tested.” Constantly emphasize that whatever goes wrong is a failure of the UI.
Even testing with one person will tell you a great deal about the problems with your UI. Testing with more people will find more issues, of course, though the law of diminishing returns kicks in surprisingly early – after testing with two or three people representing the same user group, you’ll stop getting much in the way of new information.
Paradoxically, testing with more than one person can actually reduce the number of identified problems if the test is handled badly. The reason for this lies with the team: After the first user finds problems, the team members (consciously or unconsciously) start “helping” the subsequent users. The result is that the problems found on the first test disappear on the following tests and the team decides there weren’t really any problems at all.
Fourth rule: Take the first tester’s problems seriously. Don’t help the testers or otherwise change the test for the second and third testers. Consider having a different team member supervise each test to avoid contaminating the second and third test with the results of the first test.
There’s a related problem here: expert testers. If you keep calling on the same users over and over again to do your testing, those users will get very good at working with the team’s applications. As a result, your testers will stop finding the problems that your real users will have.
Fifth rule: Keep expanding your pool of testers; avoid recycling testers
Your testers will almost certainly want to help you so, when they find a problem, they won’t tell you what the problem is. Instead, they’ll tell you what the solution is. I remember, for example, a user telling me they needed an already almost unreadable textbox to be in 8 point font. When I asked why, they said so that they could fit all the information they needed into the box. I suggested switching to a box + scrollbar instead and increasing the font to a more readable 12 point.
Instead, when doing testing, watch what your testers stumble over/hunt for/get wrong. Constantly encourage your testers to vocalize what they’re thinking about as they try to work with your UI. When your users do make suggestions for changes in the UI, probe for the problem behind the suggestion. It’s their job to find the problems; it’s your job to find the solutions.
Sixth rule: Don’t listen to your testers’ suggestions; listen to their problems.
If you follow these six rules, you’ll have something wonderful: Usability testing that finds your users’ real problems and, as a result, gives you a chance to fix them. And you won’t need a lab.
For more on UI & UI Design including usability testing, have a look at Learning Tree’s new 3-day course – UI & UX Optimized Software Design.