Should you user test with current or new users?
When it comes to which types of participants you are best to recruit for user testing, you could:
Test with people who are already customers of your product or service, or
Test with people who fit your target profile and could reasonably become users but have never used your product or service.
This choice can profoundly affect the insights you obtain from user testing, so making the right decision is essential. Recruiting existing customers is easier because you know where to find them, and current users' input is valuable. But if you don't recruit participants who have not used your product or service, you will miss out on learning what keeps people from becoming users.
Testing New Users
Testing with new users is ideal for testing first impressions of your proposition and learning how well you communicate your offering. With new customer onboarding being a critical metric for many businesses, testing with users using your solution for the first time shows you where confusion, frustration or uncertainty arise. As a result, you discover where and how to increase acquisition rates by optimising your messaging and the design of your experience.
Of course, for brand-new solutions, you can only test with potential new users, which makes sense as, initially, all users will be new, and it's essential to test the first visit.
A key challenge associated with testing new users is working out how to recruit them. Once you have defined the profile of target users, you can use social media, LinkedIn or specialist recruitment platforms like User Interviews.
Testing Existing Users
It's easy to forget that there's an inherent bias in testing designs with existing customers as they are already using the product, so they probably already like it!
Sometimes the research requires including some participants who have prior knowledge and understanding of your solution. For example, consider including current customers in your study when testing new features. If you are redesigning a solution, you should include existing customers in your testing to ensure you are not negatively affecting their experience or inadvertently removing or changing critical features on which they rely. However, running user testing sessions with existing customers needs care as they will tend to dislike design changes at first rather than focus on how easy the new design is to use. Additionally, be aware that existing customers may take longer than new users to understand a redesign as they will be familiar with how the current solution looks and works.
How we do it at Userfy
We have carried out user testing on solutions at all stages of the product lifecycle – from new concepts in the very early stages of design to well-established solutions. We recently tested a well-established B2B website with target users who had not experienced it before. This approach meant we could test how readily potential customers understood the proposition and how easily they could use the website on a first visit.
On the other hand, we also recently tested a new feature for a financial website with existing customers to test how well the new feature enhanced the current offer, how easy it was to understand and how much it was valued.
We carefully think about who we should test with for every project. Whilst we thoroughly plan our user testing to maximise the quality and value of feedback, this could be wasted if we didn't carefully consider the exact types of users to test to ensure our findings are meaningful and representative.
Can We Help You?
If there is anything we could help with, please get in touch for a completely no-obligation discussion. Even if we don't work together, we'd be delighted to advise where we can.
Phil Randall (Owner at Userfy)
Email: phil.randall@userfy.co.uk
Call: 07712669935