Remote Vs. Local User Testing

Remote Vs Local User Testing

Pros & cons to face to face user testing verses online testing.

User testing is vital for creating and maintaining a good website. User testing isn’t testing users but instead tests how they react to your website, whether they are able to complete your key tasks and how they find the overall user experience. User testing can take many forms from basic surveys to BERT tests but this post will focus on task-based user tests; where people are asked to try to perform a series of tasks on a site.

Task based user testing can be done in person, often in some kind of ‘lab’, or remotely, where the user completes the test on their own computer at home.

In lab based testing, sometimes referred to as Local Testing, a user is invited in to run through the test in a moderated room. This could be a simple desk setup in an agencies office, a more scientific lab style environment, or a lounge style lab that is set up to create a more relaxed environment for the participant. The test will be moderated by a UX professional who will give the user instructions and interact with them during the test. They will try not to lead the participant in any way but will be there to ensure they understand the requirements and to put them at ease.

Remote testing is normally unmoderated, meaning the participant follows instructions on a screen and has no interaction with a moderator. This means they are normally using their own computer in their own home. Remote testers are normally asked to ‘think out loud’ so they provide a commentary of their actions as they try to complete each task.

Remote Testing

In the past, remote testing has sometimes been labelled as a cheap and dirty option. The quality of remote testing has improved over time though; with better testing tools now available and more acceptance of it as a legitimate technique.

The main advantage of remote, over lab, testing is that it generally makes it much quicker to recruit participants and turn tests around. Online testing companies have huge databases full of testers, often broken down by various demographics, who are ready to begin testing almost immediately. With a large amount of users waiting to start it means you can run tests concurrently and 24 hours a day.

If users are broken down into demographics you can select participants that match your target audience at the click of a button. It also means you can recruit users from across the world. For example, one of our clients is targeting their product at an American audience, so it made sense for us to do the testing with American users. Remote testing will normally give you a more diverse range of users than if you are recruiting locally.

Remote users will also be comfortable in their own environment, sitting in familiar surroundings and using their own computers. If we’re testing on a Mac but the user is used to PCs then this may skew our results. This also may make testing easier for people with accessibility issues.

Testing in this fashion can be very cost effective. Companies run bespoke online user tests for around £50 per participant, which is a small price to pay for the insights that you get. As there is no lab required savings can be made here too.

Lab Testing

Lab testing is generally a bit more work but it does have its advantages over remote testing. Moderated testing means that you can keep an eye out for body language and unspoken signals that would not be clear with remote testing. You are also able to assist users and explain questions or tasks that are unclear. While you don’t want to influence how they perform tasks you can clarify any questions that they have to save them getting the wrong end of the stick.

Testing in a controlled environment means that you can dictate how the test is run by setting up the desired testing environment. This makes it easier to test on different devices and combine task-based testing with other techniques like eye tracking. It might also be possible to follow up user test sessions with focus groups using the same participants. As a facilitator, you can really observe and interact with each participant. While you don’t want to lead users in any way you can make them feel at east and encourage them to act naturally. A good facilitator will react to different situations to get the most insight from each test.

Clients can also observe these sessions which helps them feel more involved in the project, and hopefully less likely to argue with your findings!

In short, lab-based testing can be more thorough and in-depth than its remote counterpart. The key difference is that moderated testing can help you to get a better idea of ‘why’ users are doing what they’re doing. You can ask questions during the test that remove the guesswork from the process.

Getting to know the users can be an additional benefit of in-person testing. This may give you something to relate to when it comes to persona development or devising user journeys. You can learn a lot by chatting with users in person that will be useful for other stages in the development process.

Summary

So, which form of testing is better? In my opinion that depends on what you’re trying to achieve. If you’re looking to run some quick tests, or need a more diverse audience, then remote testing might be the best option. If you’re looking for more detailed in-depth results though then lab-based testing might be for you.

The purist in me says lab testing is better as you are much more involved in the process than being a passive observer. However, results from some studies tend to suggest remote testing might actually be better at uncovering user issues. This is very hard to prove either way though.

We plan to do some research of our own in this area to directly compare remote and lab testing. Keep an eye on the blog in the coming months to see the results…