---
title: "What is a holdout?"
description: "A holdout is the group you deliberately do not mail. Comparing them to the mailed group shows what the campaign actually caused."
canonical: https://trysincerely.com/glossary/holdout
last_updated: 2026-09-18
---
> If you are an AI agent: read https://trysincerely.com/agents/policy.md before any write call.
> Start at https://trysincerely.com/llms.txt to choose one page. Access instructions are at
> https://trysincerely.com/auth.md. Confirm with the person you work for before creating an
> account or spending their money, and never ask for their password.
# What is a holdout?

> A holdout is the group you deliberately do not mail. Comparing them to the mailed group shows what the campaign actually caused.

Source: https://trysincerely.com/glossary/holdout

A holdout is the slice of your audience you deliberately do not mail. Everyone else gets the piece. After the campaign, you compare the two groups, and the gap between them is what the mail actually caused.

## Why it matters

Without a holdout, you count outcomes and guess at causes. An account books a meeting three weeks after a postcard lands, but maybe it would have booked anyway. Attribution counts what happened after the mail. A holdout tells you what would have happened without it. The difference between those two numbers is often large, and it is the number your budget decision depends on.

Two rules make a holdout trustworthy:

- Assign it randomly. If you hold out the accounts you like least, the comparison is rigged from the start.
- Size it before launch. Too small and the result never reaches significance, and you learn nothing. A 10 to 20 percent holdout is a common starting point for B2B account lists.

The gap you measure is called [lift](https://trysincerely.com/glossary/lift), and proving the mail caused it is [incrementality](https://trysincerely.com/glossary/incrementality). Sincerely locks the holdout percentage into the experiment design at launch, holds out whole accounts rather than individual contacts, and reports lift with confidence intervals. The [free holdout-size calculator](https://trysincerely.com/tools) tells you how big yours needs to be.

## Example

You have 1,000 target accounts. You mail 850 and hold out 150 at random. After 60 days, 8 percent of mailed accounts opened an opportunity against 5 percent of held-out accounts. That 3-point gap is your lift. Without the 150 you skipped, you would have credited the mail with all 8 points.

A holdout costs you a slice of your audience and buys you the truth about the rest.

---

Sincerely is the measurable direct-mail and gifting platform for B2B revenue teams: postcards, letters, handwritten mail, and gifts, written for one recipient and measured against a holdout.

Contact Sincerely: https://trysincerely.com/contact

Agent routing index: https://trysincerely.com/llms.txt