One layer down, nobody writes the rules any more. This page ships 24 labelled messages, and from nothing but those labels it derives its own opinion about which words mean "spam" — that derivation is learning, and it is the entire difference between this page and the layer above. Two classical pieces do the work: TF-IDF turns text into numbers by weighting each word against how common it is everywhere else, and multinomial naive Bayes multiplies those per-word probabilities together as if every word were independent. That assumption is plainly false about English, and it works anyway.
Read that last number sceptically. The held-out set is 6 messages — a single mistake moves the score by 17 points, and "100%" on six examples is not evidence of anything. A real spam filter trains on millions and reports precision and recall separately, because sending a real message to the spam folder is far worse than missing a junk one.
Each feature's contribution is its TF-IDF weight times the log-probability gap between the two classes. Bars to the right push spam, bars to the left push ham.
—
Nothing is hidden: this is every message the classifier has ever seen. 18 rows train it, the 6 marked held out are never used for fitting.
| # | Label | Split | Message |
|---|
Each message is cut into tokens — single words plus adjacent word pairs, so "click here" is a feature in its own right — and turned into a vector of TF-IDF weights, where a term counts for more when it is frequent in this message and rare across the corpus. Multinomial naive Bayes then estimates, for every feature, how much total weight it accumulated in spam versus in ham, smoothed by adding a small constant so an unseen word never zeroes out a whole message. Classifying means adding up log-probabilities: start from each class's prior, add every feature's log-likelihood scaled by its weight, and take the larger total, with the softmax of the gap reported as confidence. "Naive" refers to the assumption that features are conditionally independent given the class, which is obviously wrong for language and famously harmless for classification, because the decision only depends on which total is bigger. Everything here is fitted from 24 examples in a few milliseconds — the algorithm is genuinely the one in production spam filters, only the dataset is toy-sized.