Over the course of the last 20 years or so, a number of academic studies have sought to determine the probability of a usability issue being encountered by single usability test participant. The probabilities determined by these studies vary, but average around 30 percent. (The classic paper on this topic, published by Nielsen and Landauer in 1993, evaluated 11 different studies and found an average probability of 31 percent.) This accordingly means that we can expect each participant in a usability test to uncover about 30 percent of all of the usability issues that affect a given website or application.
Given this, you may be tempted to think that you need only test with three users to determine 80 percent of the total number of usability issues (since 30 multiplied by three equals 90…which is more than 80). This isn’t the case, however, as there will always be some degree of overlap between the issues uncovered by users. (In the most extreme
scenario, two users might encounter the same 30 percent of usability issues, for example).
To work out how many users are required to determine 80 percent of the main usability issues that affect your site,
you need to employ the binomial probability theorem. I won’t bore you with the details of this, save to say that, when
the probability of an issue being encountered is 30 percent, you need to test with five users to reach (and
slightly exceed) our target.
In case you’re wondering, this isn’t just a theory; it’s corroborated by the results of a number of practical experiments, and a whole heap of anecdotal evidence from usability professionals too.
…But don’t start scaling back your usability tests just yet, because there are a few things you need be aware of.
Firstly, repeat after me: “Testing with five users is only appropriate in situations where all that I am interested in
is *uncovering* usability issues”. If you’re interested in *quantifying* usability issues – that is, determining how
many of your site’s users, as a whole, are likely to be affected by them – you’ll need to involve at least 20 users
in your testing. The same is true if you want to be able to generalise any of the metrics that you capture to the
behaviour of your site’s users as a whole (for example, the average time that participants take to complete a given
task; the average number of times their eyes fixate on a particular web page; or any scores that participants award
your site in terms of its ease-of-use, visual design and so forth).
Here’s why: Say that four out of five of your participants manage to successfully complete a task. Can you say that 80
percent of your users at large will be able to complete it?
Nope. You can only say (with 95 percent confidence) that somewhere between 36 percent and 98 percent of your users will be able to complete it. If you increase your sample size, however, the lower and upper bounds of this margin (or “confidence interval”) contract. So if you test 20 participants, for example, and 16 of them complete the task successfully, you can say that 58 percent to 93 percent of the larger population will be able to complete it (with 95 percent confidence) – which is a lot more accurate.
Secondly, if the scope of your evaluation is wide, you’ll need to involve more users in your test. In 2001, Spool and
Schroeder asked a group of users to purchase various different types of products, such as CDs and DVDs, at three
electronics websites. The result: The first five participants encountered only 35 percent of the usability
issues present – far lower than the 80 percent predicted by the formula described above. Why? Because the websites they were testing were large and complex, and afforded users many different ways of completing the task they had been set. Thus, if you’re going to test with five users, narrow the scope of your study. Don’t test multiple sites. Don’t
attempt a site-wide assessment. Investigate a handful of critical features and functions instead.
Thirdly, if your website serves more than a single distinct audience, you’ll need to test with five users from each. If
you have a site that is used by both children and parents, for example, you’ll need to test with five children and
five parents – otherwise you may miss critical issues that impair the performance of each group.
Fourthly, bear in mind that this “five users is enough”mantra is based on the assumption that the probability of a
user encountering a given usability problem is 30 percent.
This is a good benchmark – since, as mentioned previously, it represents the average probability, as determined from
analysis of a number of different academic studies. Some experts, however, suggest that the actual likelihood of
encountering a problem might be closer to 10 percent, or even five percent, when using a fairly well-established
website. Thus, if you suspect that your website falls into this bracket, you may want to consider testing it with more
users. (To find a problem that affects 10 percent of users, for example, you’ll need to involve a minimum of 18
participants in order to have an 80 percent chance of observing it during your test).
In summary then, five users are all you need in order to uncover 80 percent of the main usability problems that
affect your site. Don’t let the sharks of the usability world convince you otherwise!
AUTOR: @sigriffin de ETRE UK
).
el código es:
Comentarios Recientes