Know what your ads drive. Back the ones that work.
Prove what's actually driving conversions. Apply it across every campaign. Know which channels to scale and which to cut. The full measurement workflow, without the data science team or the enterprise contract.
No account. No data required. First result in under 5 minutes.
From the sample test
The platform reported
7.03x
return on spend
Holdout measured
1.90x
real return on spend
The platform claimed 18,000 conversions. 7,221 of them were incremental. Every figure on this page comes from running that test in the tool.
This is what a real answer looks like.
The result page, running the sample dataset. 8 weeks, 12 UK regions, 5 of them held out.
"What would we lose if we cut this budget?"
Every performance marketer gets asked. Most can't answer with a number. Holdout gives you a measured, defensible incrementality figure. When the question comes, you're ready.
Three tools, one loop
Use them independently or as a complete measurement workflow.
Test designer
Know which regions to hold out, whether your test has the power to detect real lift, and how long to run. Before you commit any budget.
Design a test →
Post-test analysis
Get a measured, defensible lift number with confidence intervals. Not a platform estimate.
Analyse results →
Incremental reporting
See what every campaign actually costs per real conversion. Compare channels on a level playing field. Move budget to what actually converts.
See real CPIC →
From question to defensible number
The full measurement workflow, start to finish.
Design your test
Upload your historical conversion data by geo. Holdout finds the best holdout regions, tells you how long to run, and shows what lift you can detect.
Run your campaign
Exclude the holdout regions from your ads. Everything else runs as normal. No data science team required.
Measure the lift
Upload your post-test data. Holdout measures the lift your campaign actually drove and gives you a confidence interval. Not a platform estimate.
Apply it to every campaign
Take your test result and apply it across every campaign. See what things actually cost. Know which channels to scale and which to cut.
Will a test work for you?
Two numbers decide whether a geo test can give you a clear answer: your total order volume, and how much of it the channel you want to test claims.
Use the conversions your ad platform reports for that channel.
Based on simulations of UK regional sales data. Your exact threshold depends on how your sales spread across regions. The free designer checks that with your real data.
Simple, transparent pricing
Start free. Upgrade when you're ready to go further.
Free
forever
- Test designer
- Power analysis & MDE curves
- Platform setup checklist
- Test-end reminder email
Pay per test
one payment, one complete test
- Everything in Free
- One complete test: design, activate, run, analyse
- Google Ads connected, orders from Shopify or CSV
- Holdout applied to your Google Ads account for you
- Post-test analysis with measured lift and CPIC
Growth
billed monthly, unlimited tests
- Everything in Pay per test
- Unlimited tests on supported platforms
- Always-on reporting, calibrated by your experiments
- Your numbers reconciled against platform-claimed results
- Per-channel incrementality library
Running tests for clients? Ask about an agency plan →
Presenting at Google Measurement Summit: triangulation of attribution, experimentation and MMM
Built by a practitioner
Proper measurement. Without the price tag.
I'm Jed Wrigley, a senior data scientist. I've spent years watching proper measurement teams get real answers out of marketing data. The brands that invest in it know what's working. The ones that don't are guessing.
I built Holdout to make that accessible without the headcount. It runs a real controlled experiment on your spend. Not a model, not a guess.
I come from both sides: performance marketing and data science. I know what the numbers mean and I know what the marketing team needs to hear. I've spoken on marketing measurement at Google events in London and Tel Aviv.
From question to defensible number.
Start free.
No data team. No enterprise contract. No CSV required to get started. Design your first geo test in under 5 minutes.
Sample data
Your market
Every region where you sell
Twelve UK regions. The stronger the colour, the more of your orders come from it.
The design
Some regions stop seeing your ads
A few are held back. Everything else runs exactly as before.
Eight weeks later
Every region has its own story
Sales wobble week to week for reasons that have nothing to do with ads. No single region tells you anything.
Two groups
Each group adds up to one line
Left, the regions that kept their ads. Right, the ones held back. The wobble cancels out.
Side by side
Now put them on the same axis
They tracked each other for months before the test. That is what makes the comparison fair.
The answer
The space between them is what the ads did
The held-back line is what would have happened anyway. Everything above it, your ads caused.
Now do it with your sales.
Every account gets its own holdout. Connect your shop and we read your orders, or drop them on the map.
We read your orders straight from the store. Nothing to prepare.
Read-only. We keep weekly regional totals, never your orders. Disconnect any time.
Or : postcode, date and orders. A standard shop export works. Drag it onto the map or click.
Designing your test is free.
Your ads drove 11% more orders.
You were working with 7.03x. The experiment measured 1.90x. Compare that against the return this channel needs to earn its budget.
Your real return on spend
1.90x
What the platform reported
7.03x
The rest of the correction
Cost per incremental order
£8.86
platform said £3.56
Revenue incrementality
27%
of claimed conversion value was incremental
Incremental orders
+7,221
over 8 weeks (+903/week)
Lift
10.6%
range 9.8% to 11.5%
Of what the platform claimed
40%
range 37% to 43%
Incremental revenue
+£121,702
range £111,948 to £131,455
Revenue rose 7% against 11% more orders. The orders these ads brought in were worth less than your average order.
What happened, against what would have happened
The control blend tracks your treatment regions before the test starts. After the holdout goes live, the gap between the two lines is the effect. The dotted line adds what the platform claimed it delivered, averaged across the test weeks.
Apply this to everything you spend
Correct every campaign row with this measured rate, and keep it corrected between tests.
Test another channel
One rate covers one channel. Brand search, shopping and prospecting each behave differently.
How it was measured
Synthetic difference-in-differences. A blend of your holdout regions is fitted to your treatment regions before the test, then used as the counterfactual after it.
How closely the blend tracked before the test
92%. Above 90% is a strong fit. Below 70% means the result is directional only.
What the range was measured against
Measured against how much your own regions drift in windows where nothing changed, taken from your history before the test.
Precision achieved
Plus or minus 0.9 percentage points around the estimate.
Test window
8 weeks after the holdout went live, against 25 weeks of history before it.
Measured window
1 Jan 2024 to 25 Feb 2024, inside the declared test dates 31 Dec 2023 to 26 Feb 2024. The declared start date is treated as the day exclusions were applied, so it is left out. The declared end date is treated as the day exclusions came off, so it is left out.
Regional shocks
Every region's test-window behaviour sat inside its own historical range. We checked; nothing moved your numbers.
What the platform claimed
18,000 conversions and £450,000 conversion value on £64,000 of spend for the tested campaigns in this window.
Rows matched to UK regions
All 4,114 rows in your data matched a UK region.
What this assumes
That nothing else changed in your treatment regions during the window, and that the tested channel's spend and claimed conversions sat almost entirely in those regions because it was switched off in the holdout.