WEIRD Bias

What lies behind it

In 2010, Henrich, Heine and Norenzayan coined the acronym WEIRD: Western, Educated, Industrialized, Rich, Democratic. They showed that many findings in behavioural research are based on samples from such societies, which in important respects are particularly unrepresentative of humanity.

Language models inherit this imbalance from their training data. Atari and colleagues found that model responses most closely resemble people from WEIRD societies and that the similarity decreases markedly with growing cultural distance from the USA. The relationship was strong: r = -0.70 across 49 countries.

Why this matters for your decision

From a European perspective, WEIRD at first sounds like ourselves. But the reference point is usually the USA. German, French, Spanish or Scandinavian consumers differ in price consciousness, trust in brands, attitudes to contracts and data protection. A model that matches US patterns better than European ones introduces a bias into a market entry in Spain or a tariff in Austria that is not visible in the result.

This is particularly relevant for multi-country decisions: if differences between markets appear too small, the choice easily falls on a single solution that does not hold up in individual countries.

In addition, not all groups within a country are equally well represented in text. Older people, rural regions or people who write little online leave fewer traces from which a model can learn.

How to deal with it: with synthetic results, ask what data the model relies on for your market. Use synthetic respondents for hypotheses about country differences, and check decisions on market entry and pricing in each country with real people.

Example

A consumer goods manufacturer is evaluating a refill pack in Germany, France and Italy. Synthetic respondents show similar agreement of 58 to 61% in all three countries. A behavioural test with real people in all three markets, with the same offer and the same target group definition, reveals clear differences in measured purchase intent between the countries.

For the question of which market to launch in first, precisely this difference is the information.

How it differs

WEIRD bias describes whose patterns a model reflects. Variance compression describes how uniformly it responds. The two often occur together. WEIRD bias differs from the sampling error of traditional surveys in that it does not decrease with more simulated cases.

Limitations

The key study on language models is available as a preprint, and newer models are being deliberately trained on more languages and cultures. The extent may therefore change. The question of how well a model knows your specific market nonetheless remains valid.

Behavioural tests have their own limitation: they measure real people in the market where the ads run and cannot simply be transferred to other countries. That is why Horizon measures each market separately for multi-country questions.