Attribution dashboards and holdout tests

What a dashboard does

Every ad platform reports the orders it touched. Add them up across platforms and the total is usually more than you sold, because the same order was touched by several. An attribution tool, whether it is last click, data-driven or a paid multi-touch product, takes the orders you actually had and divides the credit among the touches according to a rule. Different rules produce different splits. None of them changes the pile being divided.

That is the limit. A dashboard can tell you that search touched 60% of your orders and Meta touched 40%. It cannot tell you how many of those orders would have happened with no ads at all, because every order it can see is an order that happened. The counterfactual is not in the data.

Why the arithmetic cannot rescue it

There is a tempting shortcut: if the platforms together claim 130% of your revenue, discount their claims until they add to 100. It fails twice. It only says anything when the claims exceed the total, so a set of platforms claiming 80% between them looks fine even when half of that is not incremental. And when it does fire, it understates the gap, because it stops at the total and the true incremental share can sit well below it. Benchmark multipliers borrowed from other advertisers fail the same way, with someone else's numbers.

The only way to see the counterfactual is to create it: keep the ads off for some customers and on for the rest, and compare total sales. In a market where individual-level holdouts are unreliable, regions are the unit. That is a geo holdout test.

What the difference looks like in numbers

The same channel, the same weeks, the same 18,000 orders on the dashboard. The holdout measured 7,504 of them as orders that would not have happened without it. The dashboard was not wrong about what it counted. It was counting the wrong thing for the decision.

The direction is not always down. Channels that work early in a purchase, prospecting and video especially, are routinely credited with less than they earn, because the last touch belongs to someone else. A holdout measures those above 100% of their claim. The dashboard cannot see that either.

What a dashboard is still for

• Running the account day to day. Which ads, keywords and audiences perform relative to each other within a channel. A holdout does not rank ads; a dashboard does.
• Spotting change. A campaign whose reported numbers move is a campaign to look at, whatever the true rate is.
• Feeding the bidding. Platforms optimise on their own signal. Fix the level with a measured rate and the ranking underneath it still comes from the dashboard.

The working arrangement is to keep the dashboard and correct its level. A measured rate per channel, applied to the platform's own numbers, turns reported cost per conversion into true cost per incremental conversion across every campaign, and a re-test every few months keeps it honest as the account changes.

Which to buy

If the question is which channel deserves the next pound, a better attribution model will not answer it. It will produce a different split of the same pile. A holdout test answers it once per channel, and the measured rate does the rest of the year's work. If the question is which ad inside a channel is winning, keep the dashboard you have.