How Many Users Does it Really Take? Revisiting Usability Testing

By Timothy Dowding and Robert Johnson.

Published by The Design Collection

Format Price
Article: Print $US10.00
Article: Electronic $US5.00

In the past, it has been recommended to use 4 - 6 participants for web usability testing. This recommendation was based on limited (and at times flawed or weak) research where it was assumed that the total number of flaws in a website was an already known quantity. This research is based on two independent data sets that followed rigorous usability research methodology and techniques to determine the most cost effective number of participants required to efficiently identify the maximum number of proposed edits to a website in a cost effective manner. An initial premise is that designers are not end users and end users are not designers; therefore, the total number of edits necessary to improve a website for the benefit of the end users is an unknown. Another basis is to not include cosmetic suggestions, since they have negligible impact on the ability of a user to navigate a site. A statistical analysis determined consistency of the results across both datasets that suggest increasing the number of the number of users from 4 - 6 to 6 - 9 participants. While this may seem like a small increase in the number of participants, this has the potential to significantly impact both time and budget of a project when you consider that user testing can cost up to $2500 and a half day in time per user.

Keywords: Usability Testing, Participants, Interface Design, Discount Usability

Design Principles and Practices: An International Journal, Volume 2, Issue 2, pp.65-72. Article: Print (Spiral Bound). Article: Electronic (PDF File; 589.931KB).

Dr. Timothy Dowding

Associate Professor & Director, Operations & Information Management Department, University of Connecticut, School of Business, Stamford, CT, USA

Dr. Dowding has taught at UConn since 2001. His teaching includes both graduate and undergraduate courses in business information systems, database design, web design, and operations management. His research efforts include faculty advisor to UCONN students conducting strategic level applied business research as interns in edgelab; a collaborative program with GE within the UCONN facility at the Stamford campus. As CITI Director, Tim conducts outreach and business development for UConn; working with a wide variety of corporations to identify internship and job opportunities for students. A current initiative is the development of an international business studies program with a focus on China. He also serves on various committees, projects and grants that focus on regional economic development and workforce education. As a result of these efforts, he was awarded the UConn School of Business Outstanding Faculty Service Award for 2007. Dr. Dowding is a graduate of the U.S. Naval Academy, and retired with the rank of Commander, US Navy Reserves after 24 years of service. He was also elected twice to the Westerly RI School Committee.

Dr. Robert Johnson

Associate Profesor, Operations & Information Management Department, University of Connecticut, School of Business, Stamford, CT, USA

Robert Johnson is Associate Professor in residence in the Operations and Information Management department at UConn-Stamford. His specialty is integrating technology into managerial decision making. Other areas of expertise include qualitative and quantitative analysis of operations management problems, telecommunications systems analysis, and computer hardware and software design. Dr. Johnson has held faculty appointments at Greenville College, Penn State University, and Ohio State University. He earned a PhD in Business Administration from the University of Rochester, an MBA from the University of Pittsburgh, and a BSEE from Lehigh University. Dr. Johnson also has held engineering and economic analysis positions with the former Bell System, and spent four years as a USAF electrical engineering officer working with secure radio communications systems.

Reviews:

There are currently no reviews of this product.

Write a Review