Topic
Gpt 2
3 episodes
-
Serving Multiple Users at Once: How Continuous Batching Keeps LLM Inference Efficient MachineLearningMastery
Continuous batching is a scheduling technique that keeps LLM inference servers from wasting GPU cycles on padding. Instead of forcing short requests to wait for long ones in a fixed batch, continuous batching frees up slots the moment a request finishes and admits new work immediately, eliminating idle padding tokens and improving throughput.
-
6 Things I Learned Building LLMs From Scratch That No Tutorial Teaches You | Towards Data Science
Justy and Cody dig into what actually changes when you stop calling an LLM API and start building pieces yourself: why fine-tuning tricks like RsLoRA matter, why RoPE won, where weight tying still makes sense, why Pre-LN became the default, and how KV cache buys speed by spending memory.
-
Let’s Build the GPT Tokenizer: A Complete Guide to Tokenization in LLMs – fast
18 months ago, Andrej Karpathy set a challenge : “Can you take my 2h13m tokenizer video and translate the video into the format of a book chapter”. We’ve done it, and the chapter is below, including key pieces of code inlined, and images from the video at key points (hyperlinked to the video timestamp).