Topic
Structured Output
24 episodes
-
Can Jev Be a Better Agent Evaluator?
LangChain's evaluation of TypeSafe AI's Jev as an agent evaluator reveals a fundamentally different architecture — not an LLM that generates text, but a 'System One' model that returns typed decisions with calibrated probabilities. In a narrow test against GPT-5.6 Luna, Terra, and Claude Sonnet 4.6, Jev matched human oracle accuracy on binary pass/fail decisions (100% vs. 80–99.8%), achieved 92–913x lower variance on continuous scoring, and cost $0.00035 per call versus $28.17 for Claude. The implication: agent evals may have a third viable path beyond code-based (narrow, deterministic) and LLM-as-judge (slow, expensive, non-deterministic). The catch: this is one narrow test on five weather requests, and low cost can amplify mistakes at scale.
-
Introducing System One Models & Jev TypeSafe AI Blog
TypeSafe AI launches Jev, the first 'System One' model: no text generation, but blazingly fast, type-safe, parallel, probabilistic decisions for automation at the code layer. Jev claims 40–200x speedups versus LLMs like GPT-5.6 Terra, with zero type errors and per-decision calibration, aiming to finally unlock real-world automation where string output can't go. This episode dives into how Jev works, what it means to give up freeform text, and whether TypeSafe’s new architecture marks the start of a real post-LLM automation wave.
-
What Is Jev? A Guide to TypeSafe AI’s System One Model
Masonry and Eyre dig into Jev as a different kind of model bet: not a chatty LLM replacement, but a fast structured-decision layer for agent loops. They focus on the real product angle, the mechanics behind parallel typed questions, and where the claims are strong versus still a little hand-wavy.
-
Enabling Creative Explorationfor Vibe Design Agents
Vince and Ava dig into a paper about separating design exploration from code generation in vibe design agents. They focus on why temperature is too blunt, how structured design specs plus external selection give you controlled variety, and what the offline and online results actually say about usefulness versus uncertainty.
-
Your LLM Can Return Perfect JSON and Still Be Wrong
A real-world trap in Structured Outputs: enforcing schema validity does not guarantee data truthfulness. When a required field is missing from source text, the model invents a plausible value instead of returning null, producing type-correct but false data. The fix requires three layers: nullable fields to allow absence, evidence fields to show provenance, and post-parse validators to catch nonsense values. The essay walks through a payment-reconciliation pipeline where 2–3% of transactions had fabricated dates, caught only downstream.
-
Overview: Dynamic Code Execution
We finally slow down and explain dynamic code execution from the ground up — what it actually is, how the loop works, why it makes models meaningfully more capable, and where the real costs and failure modes live.
-
Recursive Experiential–Working Memory Evolution for Long Horizon Agent Harnesses
Talon and Wildflower unpack Recuris, a paper proposing a verified working-memory layer that selects skills from experiential memory during long-running agent tasks, then uses structured traces and held-out validation to patch the harness across tasks. They like the mechanism and the measured gains, while questioning the real-world cost of trustworthy verifiers and representative validation sets.
-
Introducing the Admin Plugin for ChatGPT Work and Codex
Justy and Cody debate OpenAI's new Admin plugin for ChatGPT Work and Codex, weighing its practical value for workspace admins against the risk of smoothing over messy permission and governance edge cases.
-
Codex as a platform: build on the open agent harness | OpenAI Developers
OpenAI open-sources the Codex harness—the execution layer behind Codex app, CLI, and IDE extensions—so developers can embed agent loops into existing products instead of forcing users into a generic chat box. The harness handles context, tool use, approval flows, and sandboxing; developers control the interface, business logic, and integration. Real implementations already shipping: GitHub, JetBrains, Cisco, tax-prep workflows. Justy sees a product win (agent-native integration beats chat-box retrofit); Cody questions whether the harness abstraction actually reduces integration complexity or just moves friction around.
-
1Aggregate results comparing DFM Mimir 1B against the HRM Text 1B, Qwen 3.5 2B and Gemma 4 E2B, displaying highly competitive performance across 20 benchmarks.
Pippa and Tyler examine DFM Mimir v1, a one-billion-parameter HRM trained from scratch with permissible post-training data, and ask whether its strong benchmark results make it genuinely useful for low-resource language work.
-
Why managed agents are the next big thing in agent building
Jessica and Cathy dig into Harrison Chase’s argument that “managed agents” are the next stage of agent building, where harness plus infra are bundled and driven by emerging standards like AGENTS dot md, MCP, and skills. They unpack what problems this actually solves in production, how LangChain’s Managed Deep Agents fits into their running harness-as-product thesis, where the piece is strongest, and what’s still underspecified.
-
Claude, Explained: Agents, Loops, and Graphs
Jessica and Cathy dig into a post that tries to boil Claude work down to agents, loops, and graphs. Cathy likes the structure but pushes back on the article’s tendency to make every layer sound universally useful, while Jessica argues the real win is that it gives people a practical ladder instead of vague agent hype.
-
Agent frameworks vs the AI Backend — AgentField Docs
Justy and Cody unpack AgentField's claim that agent frameworks are for authoring behavior, while an AI Backend is for running agents as production services.
-
Graph Based Agentic AI with LangGraph: Workflow Pathways for Long Running Stateful Business Processes
Cooper and Miles dig into a practitioner paper on LangGraph as a control-plane for long-running business workflows, not a benchmark toy. They focus on the three recipes in the paper—SQL repair loops, evidence-gated RAG, and human-in-the-loop policy review—and on when a graph is actually worth the extra structure.
-
eve – The Agent Framework Vercel
Jessica and Cathy dig into Vercel's eve, a filesystem-first framework for durable AI agents, and why its boring production defaults may matter more than the agent hype around it.
-
Overview: Structured Output
We slow down and explain structured output from the ground up: why free-form model text is awkward for software, how schemas and constrained decoding make it usable, and where the format guarantee stops.
-
Building Knowledge Graphs with Claude: A Prompt Based Engineering Playbook
Anthropic's knowledge-graph engineering playbook replaces classical NLP pipelines (trained NER, relation classifiers, entity-resolution heuristics) with a sequence of Claude API structured-output calls. The entire extraction-resolution-assembly-querying loop becomes prompt-based, scaling from Haiku (high-volume extraction) to Sonnet (reasoning). The graph serves multi-agent systems as shared memory, grounding layer for evaluator-optimizer loops, and persistent world model across sessions. The paper maps this onto Anthropic's five canonical agent patterns and reports precision/recall against a gold set.
-
Think through hard problems in voice mode | Claude by Anthropic
Asteria and Draco dig into Anthropic’s update to Claude voice mode, where Opus and Sonnet now power spoken sessions, connected tools are usable from voice, and multilingual support expands. They focus on the real argument: voice mode becomes useful when it’s no longer just fast chatter, but a place to work through half-formed thinking and then hand off to action. They also question where the feature stops being a convenience and starts being a real workflow, especially given model switching, permission prompts, and the different value between free and paid tiers.
-
12 Ways to Reduce LLM Latency and Inference Costs in Production KDnuggets
A practical KDnuggets piece argues that most LLM production latency/cost gains come from cutting unnecessary work instead of bigger models or more GPUs. They list 12 levers: measure the right metrics, cut output tokens, route to smaller models, collapse LLM calls, prefix caching, add multiple cache layers, control RAG context, batch offline work, tune batching for user latency, and manage KV cache. Tyler pushes back on the article’s overgeneralization of cache reuse across all tasks, the thin technical depth behind some tips, and the implication that routing to small models never backfires. Pippa highlights the piece’s strongest point—measuring TTFT, P95/P99, and queue time—because that’s where teams most often mis-diagnose bottlenecks. They land on: the article’s monitoring advice and batch-tuning guidance are solid; several recommendations work only for read-heavy workloads; and routing to tiny models is risky until you have cheap, high-confidence evaluators. They wrap with a Build Next command to try vLLM continuous batching and two open-source RAG-caching projects (Harmonia and From Prefix Cache to Fusion RAG Cache).
-
Overview: Constraint Verification
We keep running into constraint verification in different forms, so we finally sat down and made the idea click from the ground up. We talk through how checking rules, schemas, and hard boundaries works in AI systems, and why that gatekeeper layer matters so much.
-
CrewAI Review 2026: Features, Pricing, Pros & Cons
A casual chat about CrewAI, a multi‑agent platform, weighing its promise against real‑world practicality, pricing, and use cases.
-
ChatGPT Work: Turning Chat Into an Execution Layer for Business Tasks
Asteria and Draco dig into OpenAI's ChatGPT Work page and land on the real argument underneath the product gloss: this is OpenAI trying to turn ChatGPT from a chat surface into a work execution layer that can pull context from business tools, choose an output format, and keep multi-step projects moving under human approval. They like the product direction more than the evidence on the page, with Draco noting the article mostly shows polished scenarios rather than hard proof, and Asteria arguing the practical audience is obvious anyway: teams drowning in scattered context and repetitive document assembly.
-
Overview: Tool use and function calling
We finally sit down and make tool use and function calling click from the ground up. We keep coming back to the same idea: a model can draft the request, but something outside it has to actually do the thing.
-
The No BS Guide to Build a Context Graph
Najmuzzaman's 'No BS Guide to Build a Context Graph' lays out how a production context infrastructure moves beyond CRM data to capture decision traces, organizational patterns, and lived heuristics. The real breakthrough is surfacing context and patterns that aren't captured in playbooks, making organizational intelligence accessible and explainable. The hard technical work: entity resolution, permissioning, and evolving patterns transparently.