Data Alone Doesn't Make Delightful Experiences
Knowing that only 40% of the participants are able to complete a task doesn't say why users had trouble with that task or how to make it easier.
Originally published on February 14, 2020
Spend some time on Twitter and LinkedIn, and you'll start to notice a lot of designers and product folks talk about being data-driven.
But if you spend some time talking to them, you'll realize that all they actually do is track and collect tons of user data, and then use it to A/B test their way to increase conversion.
From a strict marketing point of view, conversion-driven iteration is fine. But then, what I've noticed in many of our clients is that, very soon, variables like churn rate and the always elusive engagement start to go astray.
Disclaimer: All links to products are Amazon affiliate links.
The problem with the conversion-driven approach
A high-quality digital product should do more than increase sign-ups. It should be informative, easy to learn, and pleasant to use.
An easy to use product not only makes people want to use it more, but it also requires less customer support, which means you can invest more in marketing or engineering. Heck, you might even be able to afford that retreat you've been promising your team but haven't had the budget to plan.
What hard, quantitative data doesn't tell you
Quantitative data tells you what's going on, but it doesn't tell you why. You might notice the user spending a lot of time in the checkout form and dropping off. But it doesn't tell you that they want to see a conversion in their local currency before paying.
It tells you they spend a long time choosing a movie to watch, but it doesn't tell you they want to watch a Marvel movie.
Knowing that only 40% of the participants are able to complete a task doesn't say why users had trouble with that task or how to make it easier. – NN Group
Why You Need Qualitative Research
Qualitative research reveals what hard numbers obscure. And you don't even need that much of it.
Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford. – NN Group
When you understand why some parts of your product are difficult for your users, you can address them in creative ways, and craft an experience that makes them want to talk about it with their friends and family (it's the whole principle behind NPS measurements).
Conclusion
Qualitative data can point your design team in the right direction regarding specific pain points in the experience of your product. Is the difference between reading the music sheet for Beethoven's 9th Symphony and hearing it and seeing it a live orchestra perform it live.