Topic

Low Rank Adaptation

8 episodes

  1. Ep 1147

    Overview: Low Rank Adaptation

    We finally do the episode we kept promising: a proper deep dive into Low-Rank Adaptation — LoRA — the technique that lets you customize a massive model for a fraction of the cost, by learning a tiny mathematical shortcut instead of retraining the whole thing.

  2. Ep 1145

    Overview: Parameter Efficient Fine Tuning

    We finally give Parameter Efficient Fine-Tuning, or P E F T, the full Overview treatment: why it exists, how LoRA and adapters work, and why tiny trainable add-ons can steer huge frozen models. We use one worked mental model all the way through so the idea lands without assuming you already speak fine-tuning fluently.

  3. Ep 1143

    How to Train Your own Decision Model like Jev using Unsloth MachineLearningMastery

    Pippa and Tyler unpack Unsloth’s new decision-model workflow: taking an open-weight model and training it to pick from fixed options instead of generating text. They focus on why that matters for intent routing, what the FastDecisionModel setup actually does, and where the trade-off sits between a cleaner product fit and the usual questions about calibration, memory, and whether a decision head is the right abstraction.

  4. Ep 1086

    Beam is now available — Reflection

    Vince and Ava talk through Reflection's Beam becoming available as an open-weight release, focusing on what changes now that the model, serving stack, and customization tooling are all shipping together. They stay skeptical about vendor-reported benchmarks, but land on the real story being the adoption path: a model you can actually download, serve, and adapt across a surprisingly broad ecosystem.

  5. Ep 1082

    NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning | NVIDIA Technical Blog

    NVIDIA drops Nemotron 3.5 Super VL — a 120B-parameter open-weight multimodal MoE model built for specialized enterprise agents. Jessica and Cathy dig into the real argument: post-training on your own data beats the frontier on your specific task, and the efficiency story (40x inference cost reduction, 10x latency) is where the product actually lives.

  6. Ep 738

    Overview: Supervised Fine Tuning

    We finally slow down and make supervised fine-tuning click, because we keep leaning on S F T like everyone already has the whole shape of it. We build it from the apprentice-and-worked-examples picture into the actual training loop, the examples, and the trade-offs.

  7. Ep 657

    Stanford Researchers Introduce TRACE: A Capability Targeted Agentic Training System That Turns Recurrent Agent Failures Into Synthetic RL Environment

    Stanford's TRACE system converts recurring agent failures into targeted synthetic training environments, using LoRA experts and MoE routing to close specific capability gaps without retraining the whole model.

  8. Ep 556

    Introducing OpenRL: A self Hosted post training API for fine tuning LLMs | Google Open Source Blog

    Justy and Cody discuss Google’s OpenRL, a self-hosted post-training API that tries to separate RL research loops from the Kubernetes and GPU infrastructure underneath them.