Topic
Overfitting
4 episodes
-
Overview: Predictive Modeling
We finally slow down and explain predictive modeling from the ground up — the core idea that powers most of what we talk about on this show, from fraud detection to weather forecasting to the brain-prediction research we've looked at.
-
Overview: Model Generalization
We finally slow down and make model generalization click from the ground up: what it means, how you measure it, and why memorizing the training set is a dead end. We keep coming back to the same simple idea, because that’s the whole game.
-
Overview: Train Test Split
We finally slow down on train-test split, because we keep bumping into it every time we talk about whether a model actually learned something. We use the sealed-final-exam picture to make the training set, validation set, test set, overfitting, leakage, and cross-validation click without assuming ML background.
-
Better Harness: A Recipe for Harness Hill Climbing with Evals
Onyx and Echo dig into Better Harness as a claim about where agent improvement actually lives: not just in model weights, but in the loop that sources evals, edits the harness, and protects generalization. They like the practical recipe, but they keep the mechanism honest, especially where the system can overfit the optimization set or turn useful instructions into token waste.