Posted by AnswerLab Research on Jun 14, 2018 5:30:00 AM

You’ve been told you need to benchmark. But, what does that mean? Where should you start? It’s common to be unsure of how to approach a benchmarking study or to not have the bandwidth to run one. Benchmarking can be an intimidating task, but creating a benchmark is worth the trouble. You learn where you stand, where to improve, and build a clear path to long-term success.

Benchmarking studies require large numbers of participants to collect vast quantities of data, either qualitative or quantitative, and make structured comparisons to other similar products. By including large numbers of participants in these studies, you develop a data-driven foundation for understanding your product and the broader digital landscape. Benchmarking can be competitive, comparing how your company approaches a feature versus competitors or other industries. Or, it can be introspective, setting a baseline for your company to use when testing future iterations of the same product.

Benchmarking is a key component of any customer experience strategy. Here are a few different types of benchmarking studies you can utilize:

Benchmarking against your competitors

Competitive benchmarking, the most recognizable method, collects data comparing how your competitors approach a digital experience versus how your company does. This assessment can be taken with  qualitative or quantitative data measuring user interactions with your experience and your competitors’. It can be completed at any stage of your product development lifecycle. In this type of benchmarking study, you gain meaningful user perspectives on your product in comparison to your competitors’. From there, you form a data-backed understanding of your positioning within your industry and can subsequently make strategic changes to make your product more competitive in the larger landscape.

Benchmarking against yourself

A less commonly known strategy is benchmarking against yourself. Using this method, you evaluate your product at a moment in time and then run a second study at a later date, assessing your progress as you move through design iterations. When conducting qualitative research, you ask participants their thoughts on your product, discover their frustrations, see how they move through the features, and at a later date, administer the same questions to understand how your design iterations are solving user pain points.

With quantitative studies, you can dive into hard metrics, tracking time on task, success rates, quantified pathing among others, and comparing them to the same metrics at a later date. We see a lot of advantage in doing quantitative surveys like these in person so the researcher can follow up with qualitative questions and observations at the end of the study, giving you more well-rounded findings. In both of these cases, being able to compare the results of the initial benchmark to your future studies helps you monitor if your design decisions are truly making a difference to your users.

Best Practices Benchmarking

In a best practices benchmark study, data is compiled to find best practices for specific experiences across verticals. For example, if you’re trying to develop better error messages or improve alerts about your privacy policy you could look at examples of these interactions in shopping, banking, and software interactions. Because these digital experiences exist across industries, only looking at your main competitors won’t get you the data you need.

Watch participants interact with other sites, uncover what works well and what doesn’t, and gather new ideas to iterate on for future product development. Loaded with qualitative research from brands across industries, best practices studies give you a complete understanding of the digital landscape and the inspiration to take the next step in your design process. 

Keep in mind, benchmarking isn’t just scoring a report card. It’s a guide that provides actionable steps to improve your design. With outcomes that are diagnostic and prescriptive, you’ll get the insights you need to make the right changes to your product.

For an example of a best practices benchmarking study, check out our Google case study on best practices for mobile experiences. And if you’re looking to launch your own benchmarking study, get in touch!

Topics: Research, Strategy