Hypothetical Bias

Why this matters for your decision

Price questions in surveys are hypothetical: nobody pays, nobody gives up an alternative. Research shows that this pushes answers upwards. In a meta-analysis of 77 studies with 115 effect sizes, Schmidt and Bijmolt (2020) find that hypothetical willingness to pay for consumer goods is on average 21% above real willingness to pay. Older meta-analyses from environmental economics arrive at higher values: Murphy et al. (2005) at a median ratio of 1.35, List and Gallet (2001) at an overstatement by a factor of about 3.

The bias is not uniform. Schmidt and Bijmolt find it stronger for higher-value products, for specialty products and in designs where the same person evaluates several variants. Precisely where pricing decisions move the most revenue, the uncertainty is therefore greatest.

If you set a price or a price increase on the basis of stated willingness to pay, you are therefore working with an assumption whose deviation is unknown in both direction and size.

How to deal with it

Within surveys, several approaches reduce the bias: designs in which each person sees only one price, reminders of the bias before the question (cheap talk) and incentive-compatible procedures in which an answer actually has consequences. Indirect methods such as conjoint designs are not free of it either; in the meta-analysis by Schmidt and Bijmolt, the overestimation there was even higher than for direct questions.

The other route is to test the pricing decision on behaviour. In a Painted Door Test, each person sees a realistic offer page with exactly one price. What is measured is how many people choose it. In this way, Horizon compares up to six price or tariff variants with real people who arrive via Google and Meta ads. Nothing is sold.

Example

A manufacturer is planning a cordless kitchen machine. In a survey, participants state on average a willingness to pay that would comfortably support a price of €349. The behavioural test compares three prices: €299, €349 and €399. Measured purchase intent drops markedly between €299 and €349, but only slightly between €349 and €399. The price threshold is therefore lower than the survey suggests, and the gap between the two higher prices is smaller than expected.

Distinction

The Say-Do Gap is the umbrella term for the gap between statement and behaviour. Hypothetical bias is its pricing component: it concerns the level of willingness to pay. The intention-behaviour gap, by contrast, concerns whether an intention is acted upon at all.

Willingness to pay is the quantity being measured; hypothetical bias is the error that arises when it is only asked for. Social desirability can reinforce the bias, for example with premium or sustainability offers.

Limits

The meta-analyses average across very different studies. Their mean values are not suitable as a correction factor for an individual survey. There are also cases in which hypothetical and real values are close together.

A behavioural test measures purchase intent at a given price, not an actual payment. It shows how demand shifts between prices. The absolute sales volume after launch also depends on distribution, awareness and competition.