Usability testing has long been the most effective ways to gather user feedback about your website. The premise is simple: watch real users use your website, observe their experiences and iterate your site design to minimize barriers users may encounter that prevent them from accomplishing their on-site goals.
Traditionally, website usability testing is carried out in-person in a usability lab. A facilitator will sit alongside a participant at a computer and prompt them to complete tasks while observing their behavior and results “over their shoulder.”
However, over the last couple of years, we have seen a proliferation of online tools like usertesting.com and loop11.com that offer cheap, quick online usability testing conducted remotely without a physical facilitator present. The “facilitation” is done through pre-loaded prompts that appear overtop of the website that is being tested.
These services’ primary differentiator is speed and cost savings. UserTesting.com has a built-in participant pool on standby waiting to take online usability tests at $39 a pop. Provide your own subjects and the cost goes down to $29 per test. Additionally, the complexities of in-person test logistics are avoided – no rental testing facility, no travel or accommodation for the facilitator(s) or participants.
This all sounds great, right? Super quick, cheap usability testing – a marketer’s dream!
So what’s the problem with online unfacilitated testing? Let’s explore the drawbacks of this approach:
- Significant risk exists that participants provide poor quality feedback due to the absence of a real-life facilitator. Despite on-screen prompts to “think aloud” shown by the testing software, participants just aren’t used to providing commentary while navigating on their own.
- The test interface adds another layer of complexity between the participant and the website you’re testing that may blur results. Novice users may have issues navigating the test interface, understanding instructions or activating their microphones (I’ve seen my share of tests recorded with no sound!)
- Lab-based tests almost always include a webcam aimed at the participants face, capturing subconscious reactions that are not available through unfacilitated remote tests.
- Since participants typically take these tests at home, they may be immersed in distractions that may compromise results. In my experience, users at home have much less patience to see a test through to completion as opposed to tests in a lab environment where participants’ attention is more easily focused.
- Results are always more reliable if the test participants reflect the actual website audience. For pharma marketers, that means we would work to recruit test participants that are suffering from a specific condition or disease state. This process offsets somewhat the “speed” advantage that online testing websites advertise.
- Unfacilitated testing may not be suitable for websites that contain complex tools or interactions as participants may hit significant usability roadblocks that prevent them from advancing to the next task. While the identification of these roadblocks is the entire point of running usability tests, there is a risk participants cannot advance through all the tasks or become so frustrated they abandon the test altogether.
These risks are significant, yet nearly all can be compensated for through the following techniques that would be employed by your friendly neighbourhood Klick Health UXer:
- Proper participant screening, weeding out more novice users
- An increased participant pool to insulate against unusable test results due to technical issues or poor feedback – typically double that of an in-person test (about 14-16 participants)
- Tight, clearly worded test prompts that provide strong queues for participants to provide detailed, vocal feedback
In conclusion, it’s safe to assume that traditional, in-person usability testing will almost certainly yield richer findings than remote unfacilitated testing. However, there is no denying the cost-benefit that these services offer. I’d say that a properly designed remote, unfaciliated test can probably reveal 70-80% of the insights you would get in an in-person testing scenario, depending on your testing goals. After all – running any type of usability test is better than no test at all!
Get in touch with us to learn more about this exciting testing technique.