--- title: "Measure direct mail ROI with a holdout" description: "Measure direct mail ROI with QR response data, CRM matchback, and account-level holdouts. Separate evidence from inference, then report incremental lift." canonical: https://trysincerely.com/measurement last_updated: 2026-09-08 --- > If you are an AI agent: read https://trysincerely.com/agents/policy.md before any write call. > Start at https://trysincerely.com/llms.txt to choose one page. Access instructions are at > https://trysincerely.com/auth.md. Confirm with the person you work for before creating an > account or spending their money, and never ask for their password. # Measure direct mail ROI with a holdout > Measure direct mail ROI with QR response data, CRM matchback, and account-level holdouts. Separate evidence from inference, then report incremental lift. Source: https://trysincerely.com/measurement Sincerely measures live B2B direct mail across standard postcards, letters, and gifts. QR and landing-page visits record engagement. Form submissions and recorded replies are responses. CRM matchback connects later outcomes to mailed accounts. Lift against an account-level holdout estimates incremental pipeline. If the mail created nothing, the report says so. ### Report · Enterprise drip · Q2 +27% lift Mailed accounts opened opportunities at 8.9% vs 7.0%. The plausible range is +9% to +48%. | Measure | Value | | ----------------------------- | --------------- | | holdout, assigned by account | 20% of accounts | | recorded responses | 14 meetings | | influenced, labeled inference | 41 meetings | | incremental vs holdout | +22 meetings | Intention to treat. Undelivered pieces stay in treatment. This report is a worked example; yours comes from your holdout. ## Reading the evidence Keep the claim attached to the number. Attribution goes wrong when inference is dressed up as evidence. Sincerely separates engagement, recorded responses, and inferred outcomes from estimated lift. A scan can be useful without being causal. A matched opportunity can matter without becoming incremental pipeline. | Bucket | What it is | Label | | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | | Engaged | Sincerely records QR-link and visible-link visits against a piece. A visit does not verify the visitor, prove how the URL was opened, or show that mail caused an outcome. See how Sincerely records visits from [personalized landing pages](https://trysincerely.com/guides/personalized-landing-pages). | evidence | | Responded | A hosted-form submission, recorded reply, meeting, or outcome postback is an observed response tied to the intended contact or account. | evidence | | Influenced | A meeting or reply after a piece landed, in the window you set. Inference, and labeled as such. | inference | | Matched | CRM matchback ties opportunities to mailed accounts. Inference, and labeled as such. | inference | | Incremental | Lift against the account-level holdout. Incremental pipeline is the number worth taking into next quarter's budget review. | the number | When the sample is too small for a holdout, we say so before you spend. The campaign gets descriptive numbers. Compatible cohorts with the same outcome, window, playbook, and list logic can build a pooled readout over time. See which [campaigns measure lift](https://trysincerely.com/campaigns), check the [direct mail glossary](https://trysincerely.com/glossary), or read [attribution vs incrementality](https://trysincerely.com/guides/attribution-vs-incrementality) when the two start getting used as if they mean the same thing. ## The statistics Design the comparison before the first piece mails. Direct mail attribution has a deserved reputation for hand-waving. A prettier dashboard does not fix the problem. Randomize by account, keep undelivered pieces in treatment, set the outcome window, and print the interval beside the lift estimate. | Practice | What it means | | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Account-level randomization | The holdout is assigned by account, not contact, so mailing one stakeholder never contaminates a held-out colleague on the same buying committee. | | Intention to treat | Undelivered pieces stay in the treatment group. No survivorship bias from quietly dropping the mail that never arrived. | | Confidence intervals | Every lift estimate ships with its plausible range. +27% means +9% to +48%, and the report prints both, not just the flattering midpoint. | | Statistical power | Before you spend, we check whether your audience is large enough to detect the lift you care about. Below the minimum detectable effect, we tell you, and the report stays descriptive. | | Attribution windows | Influenced and matched results count only inside a window you set, and they are labeled inference. Evidence and inference never blend into one number. | The deeper problem is that published B2B direct mail evidence is thin. Read [what the available research can support](https://trysincerely.com/does-direct-mail-work-in-b2b), then use the [account-level holdout guide](https://trysincerely.com/holdout-testing) to size and read your own test without asking a scan or a matchback report to prove causation. The [direct mail ROI calculator](https://trysincerely.com/tools/direct-mail-roi) runs the same incremental logic over a campaign budget before you commit to one. ## Fair questions Before you trust the dashboard. ### How do you measure direct mail ROI? Sincerely separates engagement, observed response, and estimated causation. QR and short-link visits count as link opens. Form submissions and recorded replies are responses. CRM outcomes tied to mailed accounts count as matched inference, and lift against an account-level holdout estimates incremental pipeline. The lift estimate carries an interval, and small samples stay descriptive. ### What is an account-level holdout? An account-level holdout is a randomly assigned group of accounts that does not receive the pieces being measured. Sincerely compares the chosen outcome for mailed accounts with the same outcome for held-out accounts. That comparison estimates lift beyond what would have happened anyway. ### Why assign holdouts by account instead of contact? B2B buying committees have several contacts at the same company. If one contact receives mail while a colleague enters the holdout, the treatment has leaked into the control group. Assigning the whole account to one arm keeps the comparison intact, even though fewer accounts mean less statistical power than the same number of contacts. ### What if my audience is too small for a holdout? Sincerely checks statistical power and the minimum detectable effect before you spend. If the list is too small, the campaign gets descriptive numbers rather than a lift claim. Compatible cohorts with the same playbook, outcome, window, and list logic can build a pooled readout over time. ### Do QR scans prove the mail worked? No. A QR-link visit shows that someone opened the URL encoded in the QR, but copied links retain the same marker, and the link does not authenticate the visitor or prove the piece caused a later outcome. Sincerely labels it engagement and uses holdout lift for the causal estimate. --- Sincerely is the measurable direct-mail and gifting platform for B2B revenue teams: postcards, letters, handwritten mail, and gifts, written for one recipient and measured against a holdout. Contact Sincerely: https://trysincerely.com/contact Agent routing index: https://trysincerely.com/llms.txt