Demand Characteristics

Why this matters for your decision

In almost every study, structure, order and wording send signals about what it is about. A survey that talks about a new product for ten questions leaves little doubt about which answer would be welcome. Many participants want to be helpful, some want to appear critical. Both shift the answers away from what the same people would do in an ordinary buying situation.

For decisions about price, tariff, claim or a new offer, this means: the measured approval contains a share that comes from the study situation itself. How large this share is can hardly be determined in an individual case. It depends on how clearly the study reveals its aim, who takes part and whether anything is at stake for an answer.

Economic experiments show that the effect tends to be moderate with weak signals, but can become strong with clear hints about the study's aim. What matters, then, is how openly a study shows its aim.

Example

An insurer wants to know whether a supplementary dental tariff with immediate cover will be well received. In a concept test, respondents see the tariff description, followed by questions on appeal, novelty and sign-up intent. 38 percent say they would probably take out the tariff.

In the behavioural test, the same tariff appears as an ad and a realistic offer page. At the moment of the decision, nobody knows that a study is involved. 4 percent of visitors start the sign-up and leave their email address. The two values measure different things and are not directly comparable, but the order of magnitude shows how much approval can come from the study situation.

How it differs

Social desirability describes answers intended to look good socially, regardless of the study's aim. Demand characteristics, by contrast, align with the assumed expectations of the researchers. Acquiescence bias is a response style of saying yes, whatever the statement. Finally, the mere measurement effect describes how the act of asking itself changes later behaviour. In practice, several of these effects are often at work at the same time.

Demand characteristics differ from the Hawthorne effect in that they concern the interpretation of the study's aim, not merely the feeling of being observed.

How Horizon deals with it

In the Painted Door Test, people encounter an offer in their usual online environment, via Google or Meta ads, without a panel and without an incentive. There are no questions that reveal an aim, and each person sees only one variant. Purchase intent or sign-up intent is measured through behaviour on the offer page. Only after the decision do people learn, transparently, that it is a test.

Limitations

A behavioural test is not free of context either. The ad environment, the choice of target group and the design of the offer page shape the result. The test removes the demand characteristics of the survey situation, not every bias. Nor does it explain why someone decides. For that, qualitative interviews and surveys remain valuable, ideally after behaviour has shown what happens.