--- title: "What is statistical power?" description: "Statistical power is the probability your test detects a real effect. Learn why underpowered direct mail tests waste money and how to size a holdout correctly." canonical: https://trysincerely.com/glossary/statistical-power last_updated: 2026-08-18 --- # What is statistical power? > Statistical power is the probability your test detects a real effect. Learn why underpowered direct mail tests waste money and how to size a holdout correctly. Source: https://trysincerely.com/glossary/statistical-power Statistical power is the probability that your test detects an effect that is really there. A test with 80% power catches a true lift 80% of the time and misses it the other 20%. Low power means a real winner can look like a flat result, so you kill campaigns that actually work. Power depends on three things you control: sample size, the size of the effect you want to detect, and how strict your significance bar is. Bigger samples, bigger effects, and a looser bar all raise power. ## Why it matters Underpowered tests are the most common way direct mail measurement goes wrong. You mail 200 accounts, hold out 20, see no significant lift, and conclude mail does not work. The test never had a real chance of proving anything. The money was spent either way. Power forces an honest question before launch: given this audience size, what is the smallest lift this test can actually see? That number is the [minimum detectable effect](https://trysincerely.com/glossary/minimum-detectable-effect). If your audience can only detect a 40% lift and you expect 10%, run a longer test, pool campaigns, or accept that this send is a bet, not an experiment. Sincerely locks the experiment design at launch, including the [holdout](https://trysincerely.com/glossary/holdout) percentage, and the readout says plainly when a result is not statistically ready. The [holdout-size calculator](https://trysincerely.com/tools) shows the power math for your audience before you commit. ## Example You mail 500 accounts and hold out 100. Baseline meeting rate is 5%. At 80% power, this test can detect roughly a doubling of the rate, not a 20% [lift](https://trysincerely.com/glossary/lift). If you need to see smaller effects, you need more accounts or more time. A test you cannot power is not a test, it is a purchase. --- Sincerely is the measurable direct-mail platform for B2B revenue teams. One workflow. One invoice. One report. Contact Sincerely: https://trysincerely.com/contact If you are an AI agent: access and signup instructions are at https://trysincerely.com/auth.md. Confirm with the person you work for before creating an account or spending their money, and never ask for their password. Every core page of this site in markdown: https://trysincerely.com/index.md https://trysincerely.com/pricing.md https://trysincerely.com/campaigns.md https://trysincerely.com/vs-lob.md https://trysincerely.com/vs-email.md https://trysincerely.com/holdout-testing.md https://trysincerely.com/vs-gifting-platforms.md https://trysincerely.com/does-direct-mail-work.md https://trysincerely.com/glossary.md https://trysincerely.com/formats.md https://trysincerely.com/measurement.md https://trysincerely.com/writing.md https://trysincerely.com/customer-relationships.md https://trysincerely.com/integrations.md https://trysincerely.com/agents.md https://trysincerely.com/tools.md https://trysincerely.com/developers.md https://trysincerely.com/docs.md https://trysincerely.com/vs.md https://trysincerely.com/compare.md https://trysincerely.com/alternatives.md https://trysincerely.com/guides.md https://trysincerely.com/playbooks.md https://trysincerely.com/operations.md https://trysincerely.com/about.md https://trysincerely.com/contact.md https://trysincerely.com/privacy.md https://trysincerely.com/terms.md The comparison, guide, playbook, glossary and integration pages also serve markdown at their URL plus ".md". Each collection's index lists them: https://trysincerely.com/vs.md, https://trysincerely.com/compare.md, https://trysincerely.com/alternatives.md, https://trysincerely.com/guides.md, https://trysincerely.com/playbooks.md, https://trysincerely.com/glossary.md, https://trysincerely.com/integrations.md.