The COVID-19 crisis has had immense repercussions on most aspects of our everyday lives. For brands and marketers that focus on user experiences, that includes user testing. With social distancing restrictions, it is now no longer possible to bring participants into the lab to conduct habitual testing. This does not mean that testing must stop. In fact, remote user testing is a viable option that can be worth exploring if testing is paramount to your organization.
Remote user tests involve testing participants who are in a location physically separated from the lab. They may be left to complete the test entirely on their own or they may be assisted by a moderator who is also in a different location. As with most things, remote user tests come with their own set of pros and cons.
The pros:
You can reach more users at a lower cost, as there is no need for lab facilities, travel, etc.
Testing cycles can be much quicker and lead to results faster than when they happen in the lab
You can observe users in a more natural environment, using their own devices. This can increase the validity of your observations and your conclusions
There are fewer geographical limitations, meaning you can access more people and increase your segmentation criteria
The cons:
Tests occur in a non-controlled environment, and data quality might be compromised as participants get distracted, drop out of the test, or experience technical issues
The available tools are fewer: some lab equipment, such as eye-trackers, skin conductance recorders, and electrocardiograms now cannot be used
It can be more difficult to connect with users, read their body language, and understand their reactions to get a feeling of their subjective experience
Remote tests can still offer an abundance of information. They may utilize traditional methods, such as observation (either in real-time or through recorded videos), questionnaires, and interviews to get an idea of how users experienced a given interface. Remote user tests can also make use of eye-tracking via webcam to understand where participants are looking at any given time, and of emotion recognition technology to understand their immediate reactions to the interface and content they interact with.
If you or your organization are considering remote testing, here are 5 things you should think about:
1. Choose the right test
In times of crisis, consumers are more attuned to a company’s ability to meet their needs for empathy, care, and concern. Moreover, according to McKinsey & Company, a customer’s experience with a brand in a time of crisis can have profound, lingering effects on their sense of trust and loyalty toward that brand. As such, it is important to select a topic for your test that is customer-driven rather than profit-driven. Focus on improving an online experience or your brand experience in a way that fosters empowerment or empathy. Some highly relevant topics to consumers at this time are health and well-being, safe shopping, and financial advice.
2. Decide whether your test will be moderated or un-moderated
One aspect which will determine the type of information you can collect is whether you choose to moderate your test or not. Moderated tests are more resource-intensive but can yield richer results. Un-moderated tests are faster and cheaper, but present some methodological and qualitative challenges.
3. Consider participant compensations
One aspect which runs in favour of testing in times of COVID-19 is that many people are working from home and are already set up with computers, microphones, and webcams. It is therefore easier for consumers to participate in remote testing today than it ever was. Yet few customers will test your interface for free, especially in financially uncertain times.
If your test is short and non-moderated, you may invite customers to complete it in their free time or during a break from work by simply switching their browser windows. Tests like these can offer lower compensations. However, if your test is moderated and requires that participants connect with another person at an agreed time, provide more personal information (i.e., videos of their face, their screen, etc.) and spend more time on your test, you should consider offering higher compensation. In addition, compensation largely depends on your targeted participants. If you wish to recruit any person, you may offer a lower compensation than if you choose to recruit your own customers or trained professionals.
4. Choose the right tools
Due to its particularities and limitations, remote testing requires diligent preparation and clear objectives. Make sure you have a specific hypothesis or research question, and that they can satisfactorily be tested, tracked and answered through the tools at your disposal.
For example, you may be interested in improving the accessibility of your business’ no-contact delivery information. A remote test using traditional methods, such as a survey, may offer you the customers’ perception of how easily discoverable this information was. A moderated test with customer interviews would deepen your analysis by adding behavioural notes and more elaborated, thought-revealing responses from your participants. Adding tools such as eye-tracking and emotion recognition would allow for more profound analysis and for the discovery of precise friction points, unconscious reactions, and covert interface issues.
5. Choose the right partner
In searching for a partner, take into account the tools they use, the platforms they have at their disposal, and their capacity to offer analyses and recommendations based on the test data.
Time is also an important consideration. Should you be free of constraining deadlines, perhaps you could take on the test design, preparation and moderation yourself. If you are short on time and/or resources, however, a partner who can design and conduct tests, as well as offer complete analysis reports, may be more suitable for your needs.
Conclusion
Remote user testing is a tool worth exploring for organisations who place great focus on user experience but are unable to test during the pandemic. Although meeting consumers face to face is always a necessity, that does not mean that research must stop during social distancing. As we have shown above, remote user testing can equip you with rich, practical insights to improve and build your experiences with customers.
Want to know more about how to drive sales with remote user testing?
