February 17, 2024

When prototype testing slows you down

Overly relying on high-fidelity prototype testing slows you down. Here's what to do instead.

Tom Watkins, Tereza Sustrova

Product & Design @ Wondering

Over the past year, we've spoken to hundreds of product teams that are incorporating more evaluative user research into their product discovery process. For many of these teams, the term "user research" is used interchangeably with "high-fidelity prototype testing", the practice of testing interactive prototypes with participants that (hopefully) represent your customers.

Often, this testing is done late in product discovery and so many key decisions have already been made about how the team might address the product opportunity they’re focusing on. This approach sets you up to squander much of the potential value that could be gained from evaluative testing and so leads to teams launching a suboptimal product.

Most product teams know that cramming in research only at the end of their discovery process isn’t ideal, but are unsure how to change. Most teams also know that high fidelity prototype tests are:

Incredibly time-consuming to set up: Time and time again, we see teams spend days, sometimes weeks, developing high fidelity interactive prototypes that aspire to closely emulate how a future solution in their product might work.

High risk of wasted effort: If the tested feature does not meet user needs or expectations, the effort and resources invested in the prototype might be wasted.

Not even fully representative: Even after all that effort high fidelity prototypes are often not fully representative as some aspects are too difficult to prototype such as complex transitions or individually personalised experiences.This risks skewed or invalid learnings.

However, there is a better way.

To learn faster, test your assumptions iteratively

We believe that iteratively testing your assumptions throughout your product discovery process can save you time and lead to you shipping better products. Win win! Assumption testing has been popularised by Teresa Torres and we’d recommend reading more about it here or in her book Continuous Discovery Habits.

Making the most of assumption testing here means that instead of investing all your time into developing prototypes at the end of your discovery process, you should test your key assumptions early and often using low-effort research methods. This keeps you continuously learning and maximises your chance of investing time into building a product your users find value in and hopefully love.

To get started, identify your key assumptions as early as possible by answering questions like:

What is our riskiest assumption right now and what decisions are we trying to make?

How much information do we have about this assumption already, and do we need to learn more?

How can we break down this assumption into more manageable parts, and are all these parts equally important?

How much more can we learn easily and is that difference meaningful?

After identifying your key assumptions then pick the lowest effort and fastest method available to test that assumption. By choosing low-effort, quick-to-execute methods and repeating them throughout product development, you can iteratively minimise the risk of creating something that doesn't add value for your customers.

If not prototype testing, what methods should I use?

Begin by pinpointing exactly what you need to learn, and then select a method tailored to learn specifically that. It's important to avoid the temptation of pursuing multiple learning objectives simultaneously, as this can increase the complexity and dilute the clarity of the insights gained. To illustrate these principles and offer practical guidance, we've compiled a series of scenarios along with recommended methods.

Quickly evaluate the desirability of a new concept

The simplest way to evaluate the desirability of a concept is to interview a participant and describe the concept to them. We recommend starting any interviews by gaining a foundational understanding of your participant and validating the pain points that you’re looking to address. Secondly, make sure to test a couple of concepts as participants find it easier to compare - three is a good rule of thumb.

If it’s a very visual concept, such as a completely redesigned product dashboard, consider mocking up a static image to show the new layout of the dashboard. If you need to get feedback on a static image, image testing allows you to gather this data in a few hours.

If the thought of organising, running and analysing interviews doesn’t strike you as fast or low effort then check out AI-led interviews as they can do all that for you in a couple of hours.

Evaluate how your copywriting impacts the comprehension of your designs

Many product teams don’t have the luxury of having access to a copywriter they can rely on to help them develop the copy in their designs. Instead, copywriting in products is often written by designers on the fly.

The copy in a product has a large impact on the user experience, and it’s important to test that the copy you decide on clearly and effectively communicates what users need to know and fits with your product’s tone of voice. You might want to answer questions like:

Do users easily understand what we’re trying to convey with our copy, or do they get confused?

Are there any gaps in the information provided in the copy, and does that hinder the user’s ability to use the product effectively?

An easy way to test this is to run a simple image test where you display the image of a design containing the copy you want to test. Similarly, you can either run this as an interview yourself or let AI help you out and run the interview for you.

Keep feature discoverability tests to just one main question

Before users adopt a new feature (or product), they first need to know that it exists. Specifically assessing feature discoverability will help you learn fast if the usability of your proposed designs set your team up for success.

A great way to test this is to use in-product intercepts to recruit your own users on your website on the actual screen where you plan for your users to discover your new feature. Then, just show them the new version of that screen and capture their feedback.

Target the one screen in a flow where you know people get lost or drop off

Many companies have whole product teams that focus on increasing conversion rates across key flows in the user experience. If you’ve identified a step in the user journey that is causing customers to drop off, but don’t know why they are dropping off, a good approach is to focus your testing only on the single step of the user journey that is causing the highest drop off.

Rather than asking users to test out the whole flow in a high-fidelity prototype test, target your customers with a study at the exact step in the product where the drop off happens, and uncover the why. Alternatively, recruit users from a panel and get feedback specifically on that screen.

Adopting a lowest effort, quickest results approach to testing your assumptions will guide you towards better product outcomes faster, and increase the chance that you build a product that people love to use and find valuable. The next time you plan a prototype test, ask yourself "Is there an easier way I can learn what I want to learn?" Usually, the answer is yes.

Get started with assumption testing

We believe that scaling impactful research is important, but can also be incredibly time consuming and hard to do - particularly in a world where teams are more resource constrained than ever.

That's why we're working with teams to adopt a more impactful, iterative discovery process and developing tools that help them get there. If you want to get started testing your assumptions faster and with lower effort, launch your first AI-led user interviews and concept tests by signing up here.

Continue reading

How we’re developing a safe and responsible AI-first user insights platform

Generative AI presents an unprecedented opportunity to reconsider how you get insights about your users and build products. Here's how we're making sure that you can use it safely in your research.

The guide to multilingual user research with AI

As your product grows, a new challenge emerges: understanding the diverse expectations and needs of users across different geographies. Here's how to approach multilingual research with the help of AI.

The 5-step playbook to get started with AI for product discovery

For most teams, the product discovery process is riddled with challenges. Here's how AI can help.

Continue reading

Build better product experiences with AI-first user research