Level 6 · Metric trap: the LTV of the survivors

Survivorship bias

Survivorship bias is what happens when you study only the things that made it through a filter and treat them as typical. The classic example is planes returning from missions: the bullet holes on the survivors show where a plane can be hit and still fly home.

In customer analytics the filter is “still buying”. Any group defined as active, current or loyal has survived every month of churn so far, and it differs from a fresh customer in exactly the way that matters for value.

The numbers at Harbor & Pine

Define active as a booked order since 2025-10-01. Then:

group                  customers   revenue to date   orders
active (since Oct 1)         878         $4,714.50     8.83
not active                   862         $1,795.76     3.95
everyone who bought        1,740         $3,268.55

The active customers are worth more than everyone else because staying is what makes a customer valuable. A customer acquired tomorrow has not stayed yet. Some will become loyal regulars; many will buy a few times and leave. Valuing them as if they were already loyal overstates what they are worth, and an acquisition budget based on that number overpays for every customer.

The fix: cohorts, not survivors

Value a new customer with a group defined before the outcome: an acquisition cohort, everyone whose first order fell in a period, whether they stayed or not. Then compare cohorts over equal windows (for example, revenue in the first 365 days), as in the historic LTV level. The predictive formula from level 4 also avoids the trap, because churn is built into the lifespan.

Where else it shows up

• Satisfaction surveys of current customers: the unhappy ones already left.
• “Our average customer has been with us four years”: computed on customers who are still here.
• Product reviews written mostly by people who kept the product.
• Best-selling products this year: discontinued products are missing from the list.

Interview note

“What is survivorship bias? Give an example from business data” is a frequent question. A strong answer names the filter (“customers still active”), explains why the filtered group differs from a new one, and gives the fix: define the group before the outcome, as a cohort.