Every incrementality test needs a clean control group, and building one for ads is harder than it sounds. Ghost ads and PSA tests are the two clever answers, and one is clearly better than the other.

The problem they solve

A naive holdout just doesn't show ads to the control group. But that group is different from the exposed one in a subtle way: the exposed users were the ones the ad system chose to show an ad to, often because they looked likely to convert.

Compare "people we targeted" to "everyone else" and you measure targeting, not the ad. You need a control that was also selected, but simply didn't see the ad.

How PSA tests work

The PSA method shows the control group a placebo instead of your ad, usually a public service announcement. Both groups get selected and served something, so the comparison controls for the effect of seeing any ad at all.

It works, but it has two real costs. You pay to serve the PSAs, and the charity or filler ad can itself have some effect, muddying the clean zero you wanted.

How ghost ads work

Ghost ads are the smarter version. The ad system runs its normal auction for the control user and records that your ad would have won and been shown, then simply withholds it.

You get a control group that was selected exactly like the exposed group, logged at the moment of would-be exposure, with no money spent on placebos. That clean, cheap counterfactual is why ghost ads became the reference method where the platform supports them.

How the lift is computed

Either way, the math is the same. You compare the conversion rate of the exposed group to the control group, and the difference is the incremental lift: the conversions the ad actually caused.

  • Exposed conversion rate minus control conversion rate gives incremental rate.
  • Scaled to your audience, that's incremental conversions.
  • A confidence interval tells you whether the lift is real or noise.

What each one costs you

PSA tests cost real budget, because you buy placebo impressions, and they carry a small bias from the placebo itself. Ghost ads cost you almost nothing in spend but require the platform to support them, which not every channel does.

Both cost you the conversions in the holdout, which is the price of an honest answer.

Ghost ads and PSA tests both exist to build the clean control that attribution never has, and ghost ads do it better by logging the withheld impression instead of buying a placebo. When a platform offers them, they're how real incrementality gets measured; when it doesn't, a PSA holdout is the honest fallback.

Want a stronger data analyst role or a raise? Grab the FREE Product Analyst Playbook and get the exact roadmap to your next offer.