Product · 11 min read · 2026-10-23
Building a Customer Health Score That Predicts Churn
Most customer health scores share the same flaw: the weights were guessed in a meeting and never checked against reality. Someone decided usage was worth 40 points, support tickets 15, NPS 15, and shipped a dashboard of red, yellow and green accounts that looks authoritative and predicts nothing. The number feels rigorous because it's precise — an account is "73 out of 100" — but precision is not accuracy, and a score built on unvalidated weights is a confident guess wearing a lab coat. The fix is not more signals or a fancier model; it is derivation and back-testing.
This is a playbook for building a health score that actually predicts the outcome you care about — renewal, expansion, or churn — by choosing signals deliberately, weighting them from your own history instead of from a template, and validating against what really happened before you trust it. It is deliberately distinct from two neighbours it's often confused with: a predictive churn model, which optimises raw accuracy over interpretability, and an account-intent score, which measures a prospect's likelihood to buy, not a customer's health. Same machinery, different jobs.
What a health score is for — and what it is not
A health score is a triage tool for humans. Its job is to let a customer success team scan a book of hundreds of accounts and know where to spend attention this week, before the renewal conversation rather than after. That purpose dictates the design: it must be interpretable, because a CSM has to look at a red account and understand why it's red to act on it. An opaque score that's merely accurate fails at the one thing it exists to do — it can't be argued with, trusted, or turned into a next step. So interpretability here is a feature, not a compromise, which is exactly why a health score is not simply a churn model with a nicer colour palette.
It is also not an intent score, and the distinction is not pedantic. Account intent measures a prospect's likelihood to buy from top-of-funnel behaviour; health measures an existing customer's likelihood to stay and grow from product usage. They feed different teams at different lifecycle stages, and the signals barely overlap. Conflating them produces the classic mistake of building one scoring system and pointing it at two problems it was never shaped for. The methodology of choosing and validating weights is shared — which is why the intent-score validity guide is worth reading alongside this — but the inputs and the audience are not.
Choosing signals: three to seven, no more
Start by naming the single outcome the score predicts — likelihood to renew in the next 90 days is the most common — then pick the three to seven signals most likely to lead that outcome. Guides converge on this ceiling for a reason: beyond a handful of inputs you add noise and false precision faster than you add predictive power. The strongest signals for most SaaS products are usage-based, and within usage, breadth and core-action completion beat raw login counts. A health score should weight depth of use above session frequency, because a user who logs in daily but only touches one shallow feature is more fragile than the numbers suggest.
Round out usage with recency (how recently the account did something that matters), a support-health signal (ticket volume and sentiment), and — where your product is sold into teams — a relationship signal such as whether the economic champion is still in the account, since a departing champion can flip a healthy-looking account overnight. Define each signal with a hard threshold, not a vague band: "used a core feature on 10+ of the last 28 days," not "actively engaged." Ambiguous inputs compound into an ambiguous score. The usage signal in particular rewards being modelled as an engagement-frequency curve rather than a single count — the method in the power-user curve guide gives you a far better usage input than "days active."
Weighting without guessing (or overfitting)
The weights are where most health scores go wrong, in one of two opposite directions. The first is guessing — assigning 40% here and 15% there in a stakeholder workshop, which produces weights that reflect opinion, not the relationship between each signal and actual churn. The published templates (40% usage, 20% depth, 15% support, 15% NPS, 10% payments and so on) are fine as a starting structure but worthless as answers, because your customer base has its own structure. Derive the weights instead: look at how strongly each candidate signal correlated with churn and expansion in your own history, and weight proportionally to that observed relationship.
The opposite failure is overfitting — chasing accuracy by piling on signals and tuning weights until the score perfectly "explains" past churn, at which point it has memorised noise and generalises to nothing. The guard against both is to start deliberately simple and keep the model interpretable, reserving machine learning for after you have accumulated enough churn events to train on without the model fitting to coincidence. A crude, interpretable score derived from real correlations beats a precise one built on either guessed or overfit weights, every time. If your account base splits into very different segments, be alert to Simpson's paradox: a weight that holds in aggregate can reverse within segments, so consider scoring segments separately.
Validation is the whole game
A health score you haven't back-tested is not a health score; it's a hypothesis. The validation is concrete and unglamorous: take the score as it stood at a past point — last quarter's snapshot — and check it against what actually happened to those accounts since. The test has a blunt form worth stating plainly: if the accounts that churned last quarter were mostly scored green beforehand, the model is wrong, full stop. The same applies in reverse — if accounts you flagged red renewed and expanded with no intervention, your signals are firing on something that isn't risk. Either failure means recalibrating weights or replacing signals, not shipping the dashboard anyway.
Run this on a cadence, because the relationship between signals and outcomes drifts. A feature you ship, a pricing change, a shift in who's buying — any of these can change which behaviours predict retention, so a score validated once and never rechecked quietly decays into fiction over a few quarters. Quarterly re-validation suits most teams. And validate against a clean outcome definition: "churn" has to mean the same thing every time you test, which ties the exercise to getting your churn measured correctly — logo versus revenue, gross versus net — in the first place. You cannot validate a predictor against an outcome you haven't pinned down.
Thresholds, tiers, and acting on the score
Translate the composite into tiers only after the underlying score is validated, and make the cutoffs decisions rather than defaults. A common split is 71–100 healthy, 41–70 warning, 0–40 at risk, but the right boundaries are the ones that put accounts into buckets your team will treat differently — a tier nobody acts on differently is just decoration. Set the warning threshold where intervention still changes the outcome, not where the account is already lost, because the entire point of a leading indicator is lead time. If your "at risk" tier only lights up the month before a churn that was already decided, the score is a lagging report, not a health signal.
Then wire each tier to a specific play, because a score that doesn't change behaviour wastes the instrumentation behind it. Red triggers a human outreach; yellow triggers a nudge or an enablement touch; green frees attention for expansion motions. Track whether those interventions actually move accounts back up the tiers over time — that's the second-order validation that the score is not just predicting outcomes but enabling you to change them. A health score that predicts churn perfectly but prompts no action that prevents it has optimised the wrong objective. Build it to be read, derived from real correlations, validated against reality, and connected to a play — and it becomes the triage instrument it was supposed to be instead of a dashboard everyone learns to ignore.
FAQ
What is a customer health score?
A customer health score is a single composite number that combines several behavioural and relationship signals — product usage frequency, feature breadth, recency, support health, payment punctuality — into one indicator of how likely an account is to renew, expand, or churn. It exists so a customer success team can triage a large book of accounts at a glance, flagging who needs intervention before the renewal conversation. The key word is composite: it is not any one metric but a weighted blend, and its entire value depends on whether those weights actually track real outcomes. A health score that was never validated against who actually churned is just a confident-looking guess.
How do you choose signals for a health score?
Pick three to seven signals that are leading indicators of the outcome you're predicting, and stop there — more inputs add noise and false precision, not accuracy. The strongest signals for most SaaS products are usage-based: breadth of features used and completion of core actions, which predict retention better than raw login counts. Add recency (how recently the account was active), a support-health signal (open tickets, sentiment), and where relevant a relationship signal like whether your champion is still in the account. Define each signal with a precise threshold rather than a vague band, because ambiguous inputs produce an ambiguous score. Resist adding a signal just because you can measure it; include it only if you have reason to believe it moves with churn or expansion.
How should health-score weights be set?
Derive weights from how strongly each signal correlates with your actual churn and expansion history, not from a workshop where stakeholders argue about percentages. The common vendor templates — say 40% usage, 20% feature depth, 15% support, 15% NPS, 10% payments — are illustrative starting points, not answers; your data will have a different structure. Start simple with weights proportional to each signal's observed relationship to the outcome, keep the model interpretable, and only reach for machine learning once you have enough churn events to train on without overfitting. The danger with hand-set weights is false confidence: a precise-looking 73/100 built on guessed weights is less trustworthy than a crude three-tier flag that was checked against reality.
How do you validate a customer health score?
Back-test it: take the score as it stood at a past point and check it against what actually happened to those accounts. If accounts you scored 'green' churned at a meaningful rate, or 'red' accounts renewed and expanded without any intervention, your signals or weights are wrong and need recalibration. A useful phrasing of the test is simple — if last quarter's churned accounts were mostly green beforehand, the model is broken. Review on a regular cadence, quarterly for most teams, because the relationship between signals and outcomes drifts as the product and customer base change. Validation is not a one-time setup step; a health score that isn't periodically re-checked silently decays into fiction.
How is a health score different from a churn-prediction model or an intent score?
They answer different questions for different teams. A customer health score is a post-sale, CS-facing composite meant to be read and acted on by humans triaging accounts — interpretability is a feature. A churn-prediction model is a statistical or ML model optimised purely for predictive accuracy on the churn event, often at the cost of interpretability. An account-intent score is a pre-sale, marketing-facing signal of how likely a prospect is to buy, built from top-of-funnel behaviour, not product usage. The methods overlap — all three weight signals and must be validated — but conflating them leads to building the wrong thing: a health score that's an opaque model no CSM trusts, or an intent score dressed up as health.