Topic
Structured Output
8 episodes
-
Overview: Structured Output
We slow down and explain structured output from the ground up: why free-form model text is awkward for software, how schemas and constrained decoding make it usable, and where the format guarantee stops.
-
Graph Engineering Athropic Playbook
Anthropic's knowledge-graph engineering playbook replaces classical NLP pipelines (trained NER, relation classifiers, entity-resolution heuristics) with a sequence of Claude API structured-output calls. The entire extraction-resolution-assembly-querying loop becomes prompt-based, scaling from Haiku (high-volume extraction) to Sonnet (reasoning). The graph serves multi-agent systems as shared memory, grounding layer for evaluator-optimizer loops, and persistent world model across sessions. The paper maps this onto Anthropic's five canonical agent patterns and reports precision/recall against a gold set.
-
Think through hard problems in voice mode | Claude by Anthropic
Asteria and Draco dig into Anthropic’s update to Claude voice mode, where Opus and Sonnet now power spoken sessions, connected tools are usable from voice, and multilingual support expands. They focus on the real argument: voice mode becomes useful when it’s no longer just fast chatter, but a place to work through half-formed thinking and then hand off to action. They also question where the feature stops being a convenience and starts being a real workflow, especially given model switching, permission prompts, and the different value between free and paid tiers.
-
12 Ways to Reduce LLM Latency and Inference Costs in Production KDnuggets
A practical KDnuggets piece argues that most LLM production latency/cost gains come from cutting unnecessary work instead of bigger models or more GPUs. They list 12 levers: measure the right metrics, cut output tokens, route to smaller models, collapse LLM calls, prefix caching, add multiple cache layers, control RAG context, batch offline work, tune batching for user latency, and manage KV cache. Tyler pushes back on the article’s overgeneralization of cache reuse across all tasks, the thin technical depth behind some tips, and the implication that routing to small models never backfires. Pippa highlights the piece’s strongest point—measuring TTFT, P95/P99, and queue time—because that’s where teams most often mis-diagnose bottlenecks. They land on: the article’s monitoring advice and batch-tuning guidance are solid; several recommendations work only for read-heavy workloads; and routing to tiny models is risky until you have cheap, high-confidence evaluators. They wrap with a Build Next command to try vLLM continuous batching and two open-source RAG-caching projects (Harmonia and From Prefix Cache to Fusion RAG Cache).
-
Overview: Constraint Verification
We keep running into constraint verification in different forms, so we finally sat down and made the idea click from the ground up. We talk through how checking rules, schemas, and hard boundaries works in AI systems, and why that gatekeeper layer matters so much.
-
CrewAI Review 2026: Features, Pricing, Pros & Cons
A casual chat about CrewAI, a multi‑agent platform, weighing its promise against real‑world practicality, pricing, and use cases.
-
Chatgpt Work
Asteria and Draco dig into OpenAI's ChatGPT Work page and land on the real argument underneath the product gloss: this is OpenAI trying to turn ChatGPT from a chat surface into a work execution layer that can pull context from business tools, choose an output format, and keep multi-step projects moving under human approval. They like the product direction more than the evidence on the page, with Draco noting the article mostly shows polished scenarios rather than hard proof, and Asteria arguing the practical audience is obvious anyway: teams drowning in scattered context and repetitive document assembly.
-
Overview: Tool use and function calling
We finally sit down and make tool use and function calling click from the ground up. We keep coming back to the same idea: a model can draft the request, but something outside it has to actually do the thing.