Topic

Token Counting And Optimization

5 episodes

  1. Ep 1123

    Overview: Token Counting and Optimization

    We finally sit down and explain token counting and optimization from the ground up — what tokens actually are, why they're the unit of everything that costs money or hits a limit, and the real techniques for using fewer of them without breaking your prompts.

  2. Ep 1122

    Announcing Artificial Analysis Intelligence Index v5

    Artificial Analysis updates its Intelligence Index with more private, agentic evaluations and a new companion Progress Index. Jessica and Cathy unpack why the score is more useful for choosing models, while also becoming harder to independently audit.

  3. Ep 982

    Your Agent Is Only As Good As Your Infrastructure

    Justy and Cody dig into CoreWeave's argument that agent quality in production is mostly an infrastructure story once workflows get long, bursty, and tool-heavy. They mostly buy the core claim, but Cody pushes on how much of this is genuine systems insight versus a cloud vendor setting up next week's pitch on KV caching and routing.

  4. Ep 631

    LLM Orchestration Frameworks Compared: LangChain vs. LlamaIndex vs. Raw API Calls MachineLearningMastery

    Pippa and Tyler dig into the article’s real argument: these frameworks are not interchangeable, because each one sits at a different layer of the stack. They test the claims against production reality, especially overhead, debugging, and when abstraction stops paying for itself. The episode lands on a practical view: use the lightest layer that actually earns its keep, and don’t confuse orchestration with magic.

  5. Ep 568

    Evaluating performance and efficiency of the GitHub Copilot agentic harness across models and tasks

    GitHub Copilot's agentic harness is a single cross-experience SDK component that orchestrates tools, context, and workflow across CLI, app, and code review. The team claims it delivers task-resolution parity with model-vendor harnesses while cutting token usage across several configurations, backed by public and internal benchmarks and real-world metrics. We debate technical validity, practical stakes for teams, and whether the harness should get most of the credit.