Having Data Isn't the Same as Knowing What to Fix

We're seeing more and more companies that have invested seriously in measurement: custom attribution systems, dashboards that follow each user from the first click to the purchase, reports more detailed than any off-the-shelf tool. And still, when it's time to decide what to change on their website, all that information doesn't help as much as it should.
Ideas
Last updated:
09.10.2026

It's not a volume problem. Attribution data and behavior data answer different questions from the ones you need to ask to understand what's really going wrong in an experience.

Numbers tell you what's happening, not why

An attribution dashboard can show you exactly which step users drop off at. What it can't show you is why they drop off right there.

UX research has names for this difference: quantitative data (the numbers: how many, when, what percentage) and qualitative data (the why: what someone thought, hesitated over, or didn't understand while using your site). A number tells you something is failing. Only watching a real person use the site tells you what exactly is failing.

A simple example: seeing that mobile conversion drops right at checkout is a numbers finding. Understanding that it drops because users can't tell two active payment options apart, or because a form asks for something they didn't expect at that moment, only comes from watching someone use the site. The number was never going to tell you.

A UX diagnosis shouldn't start at the dashboard

The most common instinct, especially on teams that already have good data, is to start there: look at the report, find the exact point where conversion falls, and jump straight to redesigning that screen.

The risk is that you end up fixing the symptom the report pointed to without understanding the real cause. And the problem almost always comes back a few weeks later in a different form.

We prefer to do it the other way around: first we work out how someone using the site for the first time navigates and decides, without knowing ahead of time where it “should” fail according to the report. Then we cross-check that observation against the numbers, to confirm whether what we see in real users matches what the figures show.

How we combine data and observation in our projects

The approach that has worked best for us with clients who already have solid measurement is simple, and we apply it in the same order almost every time.

  1. Usability tests or interviews first. We watch real people use the site to form hypotheses about why things are happening the way they are.
  2. Quantitative data second. We use the numbers to prioritize which of those hypotheses would have the biggest impact if solved first.
  3. Final cross-check. We confirm that what we saw in the tests actually shows up in the behavior of all users, not just the few we observed.

Neither step replaces the other. Data confirms or rules out a hypothesis. It doesn't generate one on its own.

What this means for your business

Having a sophisticated attribution system is a real advantage, and very few companies get it set up well. But that advantage only turns into better decisions when someone sits down and, literally, watches how a person uses the site.

If you already have the data and feel it isn't telling you what to fix, you probably don't lack information. You're missing the other half of the diagnosis: the half that comes from observing real users, not from another report.

At Contra Studio we usually step in right at that point: when the data is already there, but the why is missing. If that's what's happening to you, let's talk.

FAQ: analytics data and UX diagnosis

Why don't I know what to fix on my site even though I have good attribution data?

Because attribution data tells you what is happening (which step conversion drops at) but not why. That part only comes from watching real users while they use the site.

What's the difference between quantitative and qualitative data?

Quantitative data is the numbers: how many people, what percentage, at what moment. Qualitative data is the why: what someone thought, hesitated over, or didn't understand while using the site. A complete diagnosis needs both, not just one.

Why shouldn't a UX diagnosis start straight from the dashboard?

Because you risk fixing the exact symptom the report pointed to without understanding the real cause, and the problem tends to come back in another form. It's better to first watch how someone with no preconceptions uses the site, and then cross-check that against the numbers.

In what order should I use usability testing and analytics data?

Usability tests or interviews first, to form hypotheses about why things happen. Then quantitative data, to prioritize which of those hypotheses are worth solving first based on their real impact.

‍

Suscríbete a nuestro Newsletter

 

Follow our newsletter