This is the boring, profitable centre of applied machine learning: a table of customers, a column saying who left, and a model that finds the pattern. Gradient boosting builds that model out of hundreds of almost-useless one-rule trees, each fitted not to the answer but to the error left over by all the trees before it. Below, 3,000 customers are generated from a risk formula that the model is never shown; the test of the whole thing is whether the importance bars rediscover the drivers I hid in there.
Accuracy alone is a trap here: about 73% of customers stay, so a model that predicts "stays" for everyone already scores 73% and is worth nothing. AUC asks a better question — take one churner and one stayer at random, how often does the model rank the churner as riskier? 0.5 is a coin flip, 1.0 is perfect. And because I wrote the risk formula myself, I can also tell you the ceiling: scoring customers by the true formula — the best any model could conceivably do — reaches an AUC of about 0.76, because the rest is the coin-flip noise baked into the data. A model landing near that number has not underperformed; it has run out of signal to find, which is a diagnosis you almost never get to make on real data.
train the model first
Compare the bars on the left to this list. Nobody told the model that contract type matters — it found that by repeatedly asking which single yes/no question would most reduce the error still outstanding.
| train the model first |
Boosting starts with a single constant prediction — the overall log-odds of churn — and then adds hundreds of tiny corrections. At each round it computes, for every customer, the gradient and the curvature of the loss at the current prediction (for logistic loss these are simply y − p and p(1 − p)), then searches every feature and every candidate threshold for the single split that best explains those leftovers. Split-finding uses histograms: each feature's values are pre-bucketed once, so scoring a split is a scan over 24 bins rather than 3,000 rows, which is the trick that makes real boosting libraries fast. The winning stump's two leaf values are Newton steps — gradient over curvature, damped by a regularisation term — and are added to every prediction scaled by a small learning rate, so no single stump can dominate. Repeat a few hundred times and the sum of hundreds of one-question trees becomes a smooth non-linear model, which is why this family still beats neural networks on ordinary tabular data.