Should you be user testing with existing customers or with new customers?

Making the right decision is key for maximising insights…

When it comes to user testing, you’re faced with two options:

1. Test with people who are already customers of your product or service
Or,
2. Test with people who have never used it before.

Either choice can have a profound effect on the insights you’re likely to obtain from user testing, so it’s important to make the right decision.

Testing New Users

Testing new users can be an incredibly insightful experience for revealing how people develop an understanding of your solution. Whilst your existing customers have already overcome any pitfalls in your current product, testing new users can identify issues related to customer attraction and retention that you never knew were there.

With on-boarding new customers being such a critical metric for many businesses, testing with users who are using your solution for the first time shows you where confusion, frustration or difficulties are being encountered. As a result, you discover where and how to increase acquisition rates by optimising the design of your onboarding experience.

Of course, for brand new solutions it is only possible to test with potential new users. This makes sense as initially all users will be new to the solution anyway and it’s essential to test their reaction to their first visit.

A key challenge associated with testing new users is working out how to recruit them for your research. You can’t simply dip into your current pool of customers – instead you have to consider the types of customers you’re looking to target and recruit people matching these demographics with no prior experience of your product.

Testing Existing Users

Sometimes the research requires participants to already have a good knowledge and understanding of your solution. For example, when it comes to developing new features, it’s often best to test your new design with current customers, as their prior knowledge of the solution will ensure the deepest insights are generated. If your solution is being completely redesigned, it’s also helpful to test the new designs with existing customers to ensure that you are not negatively affecting their experience and inadvertently removing or changing key features that they rely on.

Testing with existing users can be easier from a recruitment perspective as most organisations can get access to the contact details of their current customers. However, running the user testing sessions themselves with existing customers can sometimes be harder. There’s a risk that users will focus too heavily on superficial changes to the look and feel of the solution, rather than how easy it is to use. This is a natural response – particularly if they have been using the existing solution for a long time. It’s therefore important to properly facilitate the testing session to ensure users fully test the functionality of the solution, rather than just offer feedback on it’s appearance.

How we do it at Userfy

We have carried out user testing on solutions at all stages of the product lifecycle – from new concepts that are in the very early stages of design, through to well-established solutions.

We recently tested a well-established B2B website with target demographic users who had not experienced it before. This meant we could test how readily a customer can use the website on their first visit. On the other hand, we also recently tested a new feature for a financial website with existing customers, to test how well the new feature enhanced the current offer, how easy it was to understand and how much it was valued.

Importantly, with every project we do we think very carefully about who we should be testing. Whilst a lot of our time is spent planning our user testing sessions so that we can maximise the quality and value of feedback we are likely to obtain – this would be a fruitless exercise if we didn’t also carefully consider the exact types of users to test to ensure our findings are meaningful and representative.

Dr Sam Howard
Meet the author:
Co-Founder and Director of Research at Userfy
07976260408

 

TALK TO US
we would love to hear about your project