--- title: "What is a holdout?" description: "A holdout is the slice of your audience you deliberately do not mail, so you can see what would have happened without the campaign." canonical: https://trysincerely.com/glossary/holdout last_updated: 2026-08-18 --- # What is a holdout? > A holdout is the slice of your audience you deliberately do not mail, so you can see what would have happened without the campaign. Source: https://trysincerely.com/glossary/holdout A holdout is the slice of your audience you deliberately do not mail. Everyone else gets the piece. After the campaign, you compare the two groups, and the gap between them is what the mail actually caused. ## Why it matters Without a holdout, you count outcomes and guess at causes. An account books a meeting three weeks after a postcard lands, but maybe it would have booked anyway. Attribution counts what happened after the mail. A holdout tells you what would have happened without it. The difference between those two numbers is often large, and it is the number your budget decision depends on. Two rules make a holdout trustworthy: - Assign it randomly. If you hold out the accounts you like least, the comparison is rigged from the start. - Size it before launch. Too small and the result never reaches significance, and you learn nothing. A 10 to 20 percent holdout is a common starting point for B2B account lists. The gap you measure is called [lift](https://trysincerely.com/glossary/lift), and proving the mail caused it is [incrementality](https://trysincerely.com/glossary/incrementality). Sincerely locks the holdout percentage into the experiment design at launch, holds out whole accounts rather than individual contacts, and reports lift with confidence intervals. The [free holdout-size calculator](https://trysincerely.com/tools) tells you how big yours needs to be. ## Example You have 1,000 target accounts. You mail 850 and hold out 150 at random. After 60 days, 8 percent of mailed accounts opened an opportunity against 5 percent of held-out accounts. That 3-point gap is your lift. Without the 150 you skipped, you would have credited the mail with all 8 points. A holdout costs you a slice of your audience and buys you the truth about the rest. --- Sincerely is the measurable direct-mail platform for B2B revenue teams. One workflow. One invoice. One report. Contact Sincerely: https://trysincerely.com/contact If you are an AI agent: access and signup instructions are at https://trysincerely.com/auth.md. Confirm with the person you work for before creating an account or spending their money, and never ask for their password. Every core page of this site in markdown: https://trysincerely.com/index.md https://trysincerely.com/pricing.md https://trysincerely.com/campaigns.md https://trysincerely.com/vs-lob.md https://trysincerely.com/vs-email.md https://trysincerely.com/holdout-testing.md https://trysincerely.com/vs-gifting-platforms.md https://trysincerely.com/does-direct-mail-work.md https://trysincerely.com/glossary.md https://trysincerely.com/formats.md https://trysincerely.com/measurement.md https://trysincerely.com/writing.md https://trysincerely.com/customer-relationships.md https://trysincerely.com/integrations.md https://trysincerely.com/agents.md https://trysincerely.com/tools.md https://trysincerely.com/developers.md https://trysincerely.com/docs.md https://trysincerely.com/vs.md https://trysincerely.com/compare.md https://trysincerely.com/alternatives.md https://trysincerely.com/guides.md https://trysincerely.com/playbooks.md https://trysincerely.com/operations.md https://trysincerely.com/about.md https://trysincerely.com/contact.md https://trysincerely.com/privacy.md https://trysincerely.com/terms.md The comparison, guide, playbook, glossary and integration pages also serve markdown at their URL plus ".md". Each collection's index lists them: https://trysincerely.com/vs.md, https://trysincerely.com/compare.md, https://trysincerely.com/alternatives.md, https://trysincerely.com/guides.md, https://trysincerely.com/playbooks.md, https://trysincerely.com/glossary.md, https://trysincerely.com/integrations.md.