Reviewing recruiting platforms this week was a useful exercise for me. Being fairly new to User Experience Design, I am unpracticed in evaluating many of these services. I have looked at many SaaS platforms for different applications, but Mechanical Turk was something I was completely unfamiliar with. It’s very easy to see how useful this service can be when time is limited for recruiting.
Conducting testing requires you to decide on a method. Face-to-face and remote testing have their own advantages depending on the resources you have available. For F2F testing, the data you get is very rich because you have the opportunity to ask questions and see body/facial expressions. These reactions are very important and F2F testing allows you to capture it during recorded sessions and while taking notes. It requires strong moderation but face-to-face testing can yield very meaningful data from just a handful of participants.
Remote testing can be effective as well but there are drawbacks. You cannot actively question participants, and many times sessions are not recorded. Some participants will also do a better job of ‘thinking aloud’ than others. Testing this way is cheaper because it doesn’t require participants to be on site or take much time, and because no researcher is needed. This allows for many tests to be taken in a short time frame.
If I were in a situation with unlimited time and resources, I think face-to-face testing is always going to be the better way to go. The reality is that many situations won’t be that flexible, and remote testing can be an effective alternative.