Primary Metric

Why this matters for your decision

If you only decide after the test which number counts, you will almost always find one by which a variant wins. With every additional measure considered, the probability rises that a difference occurs purely by chance. A primary metric set in advance protects against this and makes results comparable across tests.

For the decision, this means: management, product and insights talk about the same number, and the discussion is about what the result means, not about which metric to choose.

How Horizon sets the primary metric

The rule is: the primary metric is the last binding click before the reveal. The closer a click is to a purchase decision, the fewer curiosity clicks it contains and the stronger the signal.

Single-stage setup: the offer page contains the buy button directly, and its click is the primary metric. Two-stage setup: the offer page is followed by a product detail page; the primary metric there is the click on "Add to cart". The first click ("Learn more") is then only an intermediate signal. Whether a two-stage test is run depends on whether enough visitors can be reached within the test period.

The denominator is always the number of unique visitors to the first offer page. Ad impressions and reach are planning figures, because they depend on the delivery of the ad platform, not on the variant. The email sign-up after the reveal is a secondary signal: it measures interest in news, not purchase intent.

Example

A test compares three value propositions for a cordless kitchen machine, in two stages. Variant A achieves the most clicks on "Learn more" (11% of visitors), variant B the most clicks on "Add to cart" (2.1% compared with 1.6% for A).

Since the primary metric had been set in advance as the add-to-cart click, B is ahead. A sparks more curiosity, B triggers more purchase intent. Both findings are valuable, but only one decides.

Distinction

Secondary metrics, such as ad clicks, the intermediate signal or the email sign-up, help with diagnosis but do not decide. The click-through rate of an ad measures advertising impact, not the offer. The primary metric is also not the same as a business goal: revenue or contribution margin only arise in the market and are calculated on the basis of the test result.

Limits

A single metric condenses behaviour heavily. It says which variant triggers more purchase intent, but not why. For that, it is worth looking at the intermediate signals and, where necessary, a supplementary qualitative survey.

In addition: the further down the journey the primary metric sits, the fewer signals occur and the more visitors the test needs. The choice is therefore a trade-off between signal strength and reach, and it is documented in the test design.