Topic
Activation Function
3 episodes
-
Overview: Sparse Activation
We finally sit down with sparse activation and make the idea click from the ground up: why only part of a model wakes up on each input, how routing makes that happen, and where the real trade-offs show up. We keep it concrete, because this one has been lurking under a lot of the stuff we keep talking about.
-
Overview: State Management in Language Models
We finally do the episode we keep circling back to: state management in language models. We walk through the idea from the ground up, using the cache-and-notes picture to show why models don’t have to recompute everything every token, and where that trade-off starts biting.
-
Overview: Neural Network
We finally slow down on neural networks: what they are, how the little adjustable pieces learn from examples, and why this basic idea sits underneath so much of modern A I.