XOR is the problem that killed neural networks for seventeen years. A single layer cannot solve it — you cannot separate the corners of this square with one straight line — and that fact stalled the field from 1969 until backpropagation made hidden layers trainable. What you are about to watch is that historical moment, live: two inputs, four hidden neurons, one output, about forty lines of arithmetic and no library at all. Watch the loss sit on a plateau doing apparently nothing, then break.
The flat stretch is real gradient descent doing real work — the network is untangling four points that no straight line can separate, and the gradient through that region is nearly flat. Then the hidden units specialise and the loss falls off a cliff.
These four rows are the entire dataset. There is no test set and no generalisation claim here — the point is watching a function get fitted, one gradient step at a time.
Each forward pass is two matrix multiplications with a squashing function in between: the two inputs are multiplied by a 2×4 weight matrix to give four hidden activations, and those are multiplied by a 4×1 matrix to give one output between 0 and 1. Learning means measuring the squared error and asking, for every single weight, "if I nudged you, would the error go up or down?" — backpropagation answers that for all of them at once by applying the chain rule backwards through the network, reusing the derivative computed at each layer for the layer beneath it. Every weight then moves a small step against its gradient, and the whole cycle repeats thousands of times. The hidden layer is what makes XOR possible: each hidden neuron learns its own straight-line cut through the input square, and the output neuron combines those cuts into a shape no single line could make — which is exactly what you see in the heatmap as the boundary bends. Nothing here is symbolic or rule-based; the "knowledge" is nothing but a couple of dozen floating point numbers, which is the trade this layer of the iceberg makes — enormous flexibility in exchange for never being able to explain itself.