Test Early and Often

Posted on November 09, 2016 by Mark Stanton

Testing your product with real people offers some of the best feedback available to you when creating a website or application. Results from these user tests can supplement the assumptions, analytics data, and opinions of stakeholders that inform decisions in the design and development process. 

 

Why should I test at all?

If you're interested in finding and fixing the usability problems that exist within the product you're building, user testing is an effective and efficient way to do this.

User testing can be light on resources, inexpensive, and done at any stage within a project (the earlier the better!). Once changes have been made to the product aimed at fixing a problem, the test can be iterated so the specific interaction can be tested again to see if the changes made a positive impact.

There are other ways to identify issues, but they can have downsides that don't apply to user testing.

Examining analytics is always valuable, but truly informative analytics require lots of data before you can draw valid conclusions. This level of data is only available after a product is live and being used by real people. At this point, the team may have moved on to the next project. 

Bringing in an expert's opinion is another option for identifying opportunities for improvement. While this can be done at nearly any stage of a project, it can be expensive and may be subject to the expert's bias. In many cases, the consultant will recommend conducting a user test on your product themselves.

 

My product isn't ready yet. What should I test?

If you have a sketch of your product on a napkin, you've got something that you're able to test. While you won't be able to investigate in-depth interactions, you'll at least be able to identify if there are any glaring issues with the basic premise of your design.

To get more specific insight before spending the resources to develop your product, tests can be conducted with wireframes or prototypes. The closer these can be to the expected final product, the better, but don't avoid putting something in front of users simply because it's not finished. If the key interactions are fleshed out enough for non-project members to get an idea of what's going on, you've got enough to start basic user testing.

Numerous tools facilitate wireframing and prototyping, and choosing the right one largely depends on how detailed you want the prototype to be.

For relatively basic interactions like navigating between pages, clicking buttons, using overlays, stepping through wizards, etc., Invision provides a great tool that allows you to add "hot zones" that link together multiple static design files. This allows users to click through the designs and simulates the interactions they'd expect from a website or application.

For more advanced interactions such as conditional flows, animations, math functions, or simulating dynamic content based on user input, Axure RP offers one of the most robust tools in the industry. Axure allows for an immense amount of detail to be put into a prototype. Many users may be fooled into thinking they're using a fully coded product.

 

How do I find real users?

Testing with people that will actually use your finished product tends to provide the most accurate results. If the specifics of your project make it difficult to find and recruit these people, don't worry. Testing with less-than-ideal users is still much better than testing with no users at all.

You don't need to recruit dozens of people for your first testing session. A Nielson Norman Group study found that over 80% of usability problems can be found by testing with as few as 5 users. It's important to remember that user tests are not about proving things with scientific certainty. Instead, the goal is to improve the product you're working on by leveraging qualitative feedback.

Call on family members, friends, or random strangers on the street and ask them to spend 30 minutes to an hour with you in exchange for a free lunch or gift card. In a pinch, you can even pull in people from within your organization, but keep in mind their closeness to the subject matter could interfere with your gathering of unbiased results.

In person testing yields the best results because of the rapport that is built between the participant and facilitator. When this is not feasible, remote testing can be a suitable alternative. At minimum, you'll need screen sharing capabilities, a VoIP or phone connection, and a way to record the audio and video from the session. A webcam is recommended as well so you can see the reactions users have when they first encounter a new screen or interaction.

 

I've conducted my first user test. Now what?

Understanding your product's pain points is important, but the rubber really hits the road when you turn this feedback into actionable tasks focused on resolving the most important issues. Once you've have a solid grasp on the insights, rank the handful of newly discovered themes based on importance. Next, consider the costs associated with fixing each problem. Totally resolving each issue is the ideal solution, but reducing the issue's severity can be enough. Finally, prioritize your list of issues and fixes to determine a set of actions your team will take before your next test.

Your project team is going to be very interested in the findings and resulting action items, so be sure to present them in a way that's easy to digest. Avoid writing a research paper; condensing to a single page document or email helps crystalize the conclusions and encourages the team to engage.

 

Users tests are effective and fun! How can I learn more?

One of the most-referenced explanations of efficient user testing is Rocket Surgery Made Easy by Steve Krug. Much of the advice in this article is based on this book. A short read full of actionable tips and checklists, use this book to quickly plan and execute a successful user test. 

Check out one of Krug's user tests in action: