---
title: "Measurement"
description: "How account-level holdouts are assigned, why the experiment contract is locked at launch, and what separates the evidence in a readout from the inference beside it."
canonical: https://trysincerely.com/docs/measurement
last_updated: 2026-09-07
---
> If you are an AI agent: read https://trysincerely.com/agents/policy.md before any write call.
> Start at https://trysincerely.com/llms.txt to choose one page. Access instructions are at
> https://trysincerely.com/auth.md. Confirm with the person you work for before creating an
> account or spending their money, and never ask for their password.
# Measurement

> How account-level holdouts are assigned, why the experiment contract is locked at launch, and what separates the evidence in a readout from the inference beside it.

Source: https://trysincerely.com/docs/measurement

Measurement answers one question: what happened because of the mail, rather than what happened after it. The instrument is a randomized account-level holdout, and everything else in a readout is labelled for what it is.

## The assignment unit is the account

A holdout is assigned by account, never by contact. B2B buying committees put several people at one company, so mailing one stakeholder while a colleague sits in the control group leaks the treatment into the control and the comparison stops meaning anything.

Assignment is a stable hash of the account, the campaign and a salt. The same account in the same campaign always lands in the same arm, and anyone can recompute which arm it should be in. Nothing here uses a random number that cannot be reproduced later.

Fewer accounts means less statistical power than the same number of contacts. That is a real cost, and it is the right one to pay.

## Holdout accounts traverse the whole state machine

A held-out account is enrolled, sequenced and stepped exactly like a mailed one, with fulfilment suppressed. It is not skipped and not filtered out earlier.

The reason is symmetry. If the two arms differ in anything except whether paper arrived, the difference is no longer attributable to the paper. Keeping the state machine identical makes the readout intention-to-treat by construction rather than by promise.

Undelivered pieces stay in the mailed arm for the same reason. Dropping the mail that never arrived is survivorship bias with a tidy explanation.

## The contract is immutable after launch

The primary outcome and the measurement window are fixed when the campaign launches. Everything else in a readout is secondary or exploratory and is labelled that way.

A primary outcome chosen after the numbers arrive is not a measurement, it is a search. Locking it is what makes the interval mean what it says.

## Reading the evidence

| Bucket      | What it is                                                | Label      |
| ----------- | --------------------------------------------------------- | ---------- |
| Engaged     | A recorded QR-link or visible-link visit against a piece  | Evidence   |
| Responded   | A form submission, reply, meeting, or outcome postback    | Evidence   |
| Influenced  | A meeting or reply inside the window after a piece landed | Inference  |
| Matched     | CRM matchback tying opportunities to mailed accounts      | Inference  |
| Incremental | Lift against the account-level holdout                    | The number |

The label travels with the number wherever it goes. A scan can be useful without being causal, and a matched opportunity can matter without being incremental pipeline.

## Honest numbers or no numbers

Every lift estimate ships with its interval. A twenty-seven per cent lift that could plausibly be anything from nine to forty-eight per cent is reported as both, not as the flattering midpoint.

Before you spend, Sincerely checks whether the list is large enough to detect the effect you care about. When it is not, it says so and the campaign reports descriptively. Compatible cohorts with the same playbook, outcome, window, and list logic can build a pooled readout over time.

The words "proved" and "significant" are not used unless the claim is true. Postal events confirm delivery points, not attention.

## What to read next

[The holdout testing guide](https://trysincerely.com/holdout-testing) covers sizing and reading a test in depth, and the [holdout size calculator](https://trysincerely.com/tools/holdout-size) does the arithmetic. [What the published evidence supports](https://trysincerely.com/does-direct-mail-work-in-b2b) is the honest state of the field.

---

Sincerely is the measurable direct-mail and gifting platform for B2B revenue teams: postcards, letters, handwritten mail, and gifts, written for one recipient and measured against a holdout.

Contact Sincerely: https://trysincerely.com/contact

Agent routing index: https://trysincerely.com/llms.txt