Mere Measurement Effect

Why this matters for your decision

Asking people whether they would buy a product makes them think about that product. The question makes it more salient and can create or strengthen an intention in the first place. Studies with customer data show that respondents are then more likely to buy than comparable people who were not asked.

Two problems follow for decisions. First: anyone who checks the validity of purchase intent on surveyed people measures a validity that does not exist in this form in the wider, unsurveyed target group. Chandon, Morwitz and Reinartz call this self-generated validity. In their study, the relationship between intention and purchase was 58 percent stronger among respondents. Second: the people who later encounter the offer in the market were not asked beforehand. A demand estimate based on survey data therefore carries over conditions that are absent in the market.

Example

A bank asks existing customers whether they would switch to a new credit card with cashback. After the launch, respondents apply for the card considerably more often than customers who did not take part in the survey. The team therefore considers the survey highly accurate and extrapolates its values to the entire customer base. The extrapolation turns out too optimistic, because part of the accuracy was created by the asking itself.

How it differs

The demand effect acts on answers during the study. The mere measurement effect acts on behaviour after the study. Self-prophecy is closely related and describes how a person's own statement about future behaviour shifts that behaviour in the direction of the statement. The intention-behaviour gap describes the gap between intention and action; the mere measurement effect is one reason why this gap appears smaller among respondents.

How Horizon deals with it

In a Painted Door Test nobody is asked anything beforehand. People encounter an ad and a realistic offer page in their familiar online environment, just as they would later in the market. Purchase intent is measured through behaviour, without a panel and without incentives. Nothing is sold; anyone who decides is then told that it was a test.

Limitations

The size of the effect varies with the product and prior experience. People who already know a product react less strongly to the question. A behavioural test avoids the effect of the survey, but only measures the decision at the moment of the offer, not repeat purchase or usage. And where a survey is explicitly intended to make people aware of a topic, the effect is not a flaw but intended.