Level 2 · How sure is the mobile rate?

A rate is an estimate

The 21,549 mobile sessions are a sample of all the mobile visits Harbor & Pine will ever get. Run the same three years again and you would not get exactly 2.41% again: a few orders would land differently. A confidence interval says how far the estimate could plausibly be from the true rate, given how much data you have.

The standard error of a rate

For a share p measured on n independent visits, the spread of the estimate is its standard error:

se = math.sqrt(p * (1 - p) / n)

Two things make it small: lots of data (n is under a square root, so 4× the sessions halves the error) and rates far from 50%. For mobile, p = 0.0241 and n = 21,549, so se ≈ 0.00105, about a tenth of a percentage point.

The 95% interval

Estimates of a rate on large samples follow a normal curve, and 95% of a normal curve lies within 1.96 standard errors of its centre. So:

low, high = p - 1.96 * se, p + 1.96 * se    # 0.0221, 0.0262

Mobile converts at 2.41%, 95% CI 2.21% to 2.62%. Compare tablet: a similar rate, 2.81%, but only 3,910 sessions, so its interval is two and a half times as wide, 2.30% to 3.33%. Same kind of number, much less certainty. The bars in the chart look equally solid; the intervals show they are not.

What the interval means

The honest reading: if you repeated the whole data collection many times and built an interval each time, 95% of those intervals would contain the true rate. In practice analysts say “the data are consistent with a mobile rate between 2.2% and 2.6%”, which is fine. What it does not mean is that 95% of mobile visitors convert in that range, or that a future month will land inside it. A single month has far fewer sessions and a wider interval of its own.

Common mistakes

• Wrong n. n is the number of sessions, not the number of conversions.
• Tiny samples. The normal approximation needs enough successes and failures, roughly at least 10 of each. With 3 conversions in 40 sessions, use an exact or Wilson interval instead.
• Overlap as a test. Two intervals that overlap slightly can still differ significantly. To compare two rates, test the difference directly (level 4).
• Treating sessions as independent when they aren’t. One shopper with ten visits is not ten independent coin flips. Real tests often count users, not sessions, for this reason.