Topic
Sampling And Temperature
10 episodes
-
Can Jev Be a Better Agent Evaluator?
LangChain's evaluation of TypeSafe AI's Jev as an agent evaluator reveals a fundamentally different architecture — not an LLM that generates text, but a 'System One' model that returns typed decisions with calibrated probabilities. In a narrow test against GPT-5.6 Luna, Terra, and Claude Sonnet 4.6, Jev matched human oracle accuracy on binary pass/fail decisions (100% vs. 80–99.8%), achieved 92–913x lower variance on continuous scoring, and cost $0.00035 per call versus $28.17 for Claude. The implication: agent evals may have a third viable path beyond code-based (narrow, deterministic) and LLM-as-judge (slow, expensive, non-deterministic). The catch: this is one narrow test on five weather requests, and low cost can amplify mistakes at scale.
-
LLMs respond differently to harmful prompts when AI watermarking is used
Onyx and Echo dig into a weird side effect of text watermarking: once you change the token sampler, you can also change refusal behavior, tool calls, and how easily harmful prompts slip through. The take is that provenance is not a free add-on; it’s another moving part that has to be red-teamed like any other safety mechanism.
-
Enabling Creative Explorationfor Vibe Design Agents
Vince and Ava dig into a paper about separating design exploration from code generation in vibe design agents. They focus on why temperature is too blunt, how structured design specs plus external selection give you controlled variety, and what the offline and online results actually say about usefulness versus uncertainty.
-
How Claude's text watermarking works
Pippa and Tyler dig into Anthropic’s new Claude text watermarking: Tyler worries it’s weak, gameable, and over-claimed; Pippa pushes that even a probabilistic, low-stakes signal can still matter for platforms and compliance, and they land on it as quiet but useful infrastructure rather than a magic detector.
-
Overview: Diffusion Models
We finally slow down and make Diffusion Models click: the noisy-photo-in-reverse idea, the training loop, the denoising steps, and why this became so central for image generation. We also get into prompts, guidance, latent space, speed trade-offs, and why the concept is still very much alive.
-
Overview: Decoding Strategy
We finally slow down on decoding strategy, the rule that turns a model's next-token odds into the actual words you see. We use one hallway-and-doors picture to make greedy decoding, sampling, top-k, top-p, beam search, and newer decoding work feel less like magic knobs.
-
Overview: Sampling and Temperature
We’re finally slowing down and explaining sampling and temperature from the ground up, because we keep circling back to it and it’s one of those knobs that sounds simple until it isn’t. We talk through why the same prompt can produce different answers, how temperature changes the shape of the next-token choices, and where the trade-offs actually land.
-
Overview: Calibration
We finally slow down and make calibration click: what it means for a model’s confidence to match reality, how you measure that, and why it matters when you actually want to trust the thing. We keep it grounded in the systems we’ve been circling for ages, because calibration is everywhere once you start looking.
-
Concurrent Image Understanding and Generation: Self Correcting Coupled Markov Jump Processes
Fern and Lintel dig into a new paper on doing image understanding and image generation at the same time, inside one decoding loop. The hook is simple: most systems either describe first and draw later, or they run both sides in parallel without letting the latest text and image decisions correct each other mid-step. This paper tries to fix that with a coupled masked-diffusion sampler that can both coordinate and backtrack.
-
Overview: Autoregressive Generation
We finally slow down and make autoregressive generation click: the whole thing is just a model writing one token, then using what it wrote to choose the next one. We keep the focus on the loop, the trade-offs, and why that one-step-at-a-time setup is still the backbone of modern language models.