Usability testing refers to evaluating a product or service by testing it with representative users. Typically, during a test, participants will try to complete typical tasks while observers watch, listen and takes notes. The goal is to identify any usability problems, collect qualitative and quantitative data and determine the participant's satisfaction with the product.
To run an effective usability test, you need to develop a solid test plan, recruit participants , and then analyze and report your findings.
- Define test
- Identify subjects / Recruit participants
- Write questions
- Execute test
- Summarize results
- Identify key insights & next steps
Before you can begin to write questions or recruit for your test, you need to understand what you're studying and why. The best way to do this is to define the following items:
-
Feature/Process – What is the 1 thing you're evaluating? This needs to be a specific feature or flow; eg
user onboarding,homepage,account management, etc -
Product – Define how you’ll deliver the “thing” you’re studying to the participants. You should test 1 product at a time. eg --
mobile site,inVision prototype,desktop production,static mock-up, etc -
Test Objectives – Define a specific & clear set of goals for the test. Ask yourself: What am I trying to learn? eg --
Can users navigate to their account dashboard?orDo users understand context x on homepage? -
Business Case – Why are we doing this test? What are the benefits? What are the risks of not testing? Its important to understand the significance of our objectives. We need to be able to communicate to stakeholders the reason for commiting to this specific test
-
User Demographics – What type of user is this study focusing on? How many users? What are constraints needed on
age,gender,location,device usage, etc -
Test Method & Tools – How is this test going to be executed?
Usertesting.com?In-person with recording via quicktime?Remote via uberconference with quicktime recording? Etc
-
Hallway Testing: Grab anyone who is unfamiliar with the application or website you're testing. These can be co-workers, friends, family or folks at Starbucks. Hallway testing works well for general purpose websites or apps and when you're looking to uncover the more obvious problems with the interactions (problems, of course, always seem obvious after you find them).
Don't rely too heavily on this method, especially when the design is more refined or users have specialized skills. When users don't have an interest in a product or service they will be happy to give you an opinion on matters but relying on them may generate false positive and miss issues actual users will encounter. -
Existing Users: Your existing users are an obvious wellspring for testing. For new products, there are by definition no users, so this isn't always something you can rely on. But, if your company makes similar products, then it makes sense to leverage customers that might know the domain. Finding existing users off your website is an easy place to start. You can use pop-ups (offered by UserZoom and Ethnio) or fixed opt-in boxes to solicit volunteers. Marketing and sales departments usually have customer contact information you can tap. You can also try working with the customer support department and asking participants if they'd be interested in participating in follow-up studies. While existing customers sound like a panacea for finding users, we find not all volunteers have the time or availability to commit to an hour-long study, so you often have to rely on other sources.
-
Usertesting.com: The popular website testing service Usertesting.com not only delivers audio and video of users using a website or mobile app, it also has a large panel of users. Usertesting.com allows you to recruit based on age, gender and geography. You can also ask participants to self-select by asking them only to participate if they've used a certain website, own a particular product, have health insurance, or have a 401k account, among other things. While it's not a solution for when you need hundreds of responses or very specific recruiting criteria, getting a source of users and a 15-20 minute usability testing for under $40 is often very effective.
- Create a few tasks you want the user to perform. The task should require at least six to nine clicks to complete the task. The tasks could be signing into the system or uploading photos and setting permissions.
- Write scenarios for each task. You should explain the task without telling the user what to do. An example would be, "You were at a family picnic where you took photos of your nephews and nieces. You want to share those photos with your family on the Internet on Flickr. Where would you go to sign in, upload those photos, and set the preferences so only your family can view them?"
- List follow-up questions you might want to add.
- The magic of usability tests are the answers you get outside of the scenarios. So you might want to come up with additional questions within the screens you are testing.
- Practice.
- Check all the equipment.
- Make the space comfortable.
- Prep with your partner, if you have one.
- Friendly Introductions and housekeeping/small talk.
- Be considerate.
- Start the recording. (No seriously, start it)
- Don’t demo the product.
- Let the user do most of the talking.
- When in doubt, repeat it back.
- Leave time for their questions.
- Record notes on the session
Make sure to record notes from your session immediately after its complete. We reccomend using 1 consistent document that is bulleted by subject. This gives you one easy place to dump all your thoughts and insights for further analysis once all sessions are complete. Make sure you record everything
Don’t trust your memory; rely on your results summary that you created as you went through the test.
[Back to top]