Painted Door Test

Why this matters for your decision

Surveys show what people say. For investments in product, price or tariff, however, what counts is what they do when price, alternatives and a concrete action come into play. Meta-analyses show that hypothetically stated willingness to pay is, on average, clearly above the amount measured in real conditions. The direction and size of this gap are unknown in any individual case, which is why it can hardly be corrected without observation.

A Painted Door Test closes this gap before budget is committed. It delivers measured purchase intent per variant and thus a reliable basis for deciding between concepts, prices, features or messages.

How a Painted Door Test works

At Horizon, consumers see an ad on Google or Meta and arrive at a realistic offer page. Variants differ only in the attribute the decision is about; everything else stays identical. Up to six variants run in parallel, randomly assigned.

The decisive measurement point is the last binding click before the reveal, for example on 'Add to basket' or 'Choose tariff'. Immediately afterwards, visitors learn that it is a test and can optionally sign up to be kept informed. Personal data is collected only after this reveal. The email sign-up is a second, weaker signal. From question to data analysis takes around four weeks.

Example

An insurer wants to know which of two positionings works better for a supplementary dental tariff: 'Full reimbursement for dental prosthetics' or 'No waiting period from day one'. Both pages are identical except for the headline and benefit argument; price and benefits are the same.

Result after four weeks: 2.8% measured sign-up intent for 'No waiting period', 2.1% for 'Full reimbursement', with a 96% probability that 'No waiting period' is the better variant. The team thus knows not only the winner but also the margin.

One method, many names

Several terms are in circulation in companies for the same approach. They almost always mean the same thing: you show real people an offer that does not yet exist and measure whether they choose it.

• Pretotyping: the umbrella term, a blend of 'pretend' and 'prototype'.
• Painted Door Test: the image of a painted door that looks real but does not yet lead anywhere.
• Smoke Test: common mainly in marketing and start-ups.
• Fake Door Test: common mainly in product teams working on digital products.
• Proto-Selling: widespread mainly in the US.

How it differs

Painted Door Test, Pretotyping, Smoke Test, Fake Door Test and Proto-Selling essentially describe the same approach. There are nuances in use: a button in an existing app is more likely to be called a Fake Door Test, a standalone landing page more likely a Smoke Test. Horizon uses the terms Pretotyping and Painted Door Test.

An A/B test optimises a product that is already live and must be able to deliver every variant. A Painted Door Test checks before anything is built, outside your own systems. A pre-order test requires a deposit and creates an obligation to deliver; in a Painted Door Test no money changes hands.

Limitations

The test measures an independent online decision by consumers who can be reached through ads. It is not suitable for B2B buying committees, shelf decisions in direct competitive comparison, comparison portals or very low-priced add-on products, nor for questions about brand image or satisfaction.

It shows which variant is chosen, not why. It measures the first decision, not repeat purchase or usage. And a click is not a payment: measured purchase intent is closer to behaviour than any survey, but it is not revenue.