HoldoutA group of accounts deliberately excluded from a campaign so their outcomes can be compared against the accounts you did mail. The only design that answers what the channel caused rather than what merely followed it.
IncrementalityThe share of an outcome that would not have happened without the channel. It is what a holdout measures, and it is almost always smaller than what attribution reports.
Lift, absolute and relativeAbsolute lift is the difference in percentage points: 8.9% against 7.0% is 1.9 points. Relative lift expresses the same gap as a proportion: 27%. Absolute is what converts to pipeline; relative is what appears in marketing.
MatchbackJoining closed opportunities back to the accounts that were mailed. Useful for sizing a program, but silent on causation, because it has no comparison group.
Attribution windowThe period after delivery during which an outcome is counted as related to the piece. Longer windows credit the channel with more, which is why the window has to be fixed before the campaign runs.
Intention to treatAnalyzing every account as randomized, including those whose mail never arrived. Dropping undelivered pieces looks like data hygiene and reliably inflates the result, because bad addresses cluster in dead accounts.
Minimum detectable effect (MDE)The smallest lift a given design could reliably find. Computed from base rate, sample size and power. A program whose MDE is larger than any plausible effect cannot learn anything, however it is reported.
Statistical powerThe probability a test detects a real effect of a given size. Conventionally 80%. Low power produces null results that get misread as evidence the channel failed.
Confidence intervalThe plausible range around an estimate. A 27% lift spanning +9% to +48% is a finding; the same 27% spanning −5% to +61% is noise. Reporting the midpoint alone hides which one you have.
Direct responseA response traceable to the piece itself — a QR scan or a landing page unique to that mailing. Evidence rather than inference, and it needs far less volume to measure, because the baseline is zero by construction.