How to Debug Coding Agents with LangSmith Traces
We dig into LangSmith's new push to unify observability for multiple coding agents in one place. Cody examines whether a single trace schema can survive real heterogeneity and what still leaks through. Justy talks to who this actually helps and where teams are likely to run before they bother. One parsing bug, one shared laugh, and a concrete demo of why 'diff-only debugging' is a trap.
Transcript
Justy I burned maybe ten tokens on that pagination subagent and still shipped the wrong pagination. It didn’t even leave a footprint in the diff—just a green check that made me wonder what the hell happened.
Cody Yeah, because the diff is the last ten seconds of the show. The trace? It’s the whole episode.
Justy Exactly. We had one little cursor-based spec, the agent spawned off an offset-based helper from six months ago—no mention in the final diff, just a quiet incorrect branch.
Cody MCP doesn’t always surface the tool call, so the trace tree swallows the typo. That’s the leak you’re chasing—obvious mistake hiding behind a pretty JSON schema.
Justy Right—and I don’t want to care which agent I’m using today. Yesterday it was Cursor, today it’s Codex, tomorrow it’s Claude Code, and every time I have to remember a new mental model to debug. That’s not infra, that’s a zoo.
Cody So LangSmith sells a single layer: same trace fields, same query syntax, same dashboard, no matter which coding agent wrote the code. They map Claude Code, Codex, Cursor, Copilot Chat, Pi, OpenCode, and DeepAgents Code into one canonical schema.
Justy So even if the underlying agent emits OpenTelemetry shaped like a snowflake, I still see user turns, assistant turns, model calls, tool calls, shell commands, MCP activity, subagent invocations, errors, retries, timing, tokens, cost—all in one place.
Cody Exactly. And when you zoom in, you find the exact moment the test failed, the agent read the assertion message, and edited the file instead of re-running—mutation, not iteration. The diff just shows the green check at the end.
Justy That’s the hidden shortcut you miss if you swear by diffs only. Traces show the scaffolding that wasn’t torn down.
Cody So the loop is reconstruct, debug, improve. Reconstruct the sequence, filter by thread_id, find the failure path, then turn the mistake into a rule—Skills you can run across every agent.
Justy Which means your eval suite grows the moment you save the bad run. You can prove the fix holds and catch regression later. It’s not prompt tuning, it’s session tuning.
Cody All good until the MCP layer decides to ghost the tool call from the trace tree. Their schema captures the start and end of the MCP invocation, but inputs, outputs, and token counts can still vanish.
Justy So they haven’t solved the black box at MCP level. But at least the coding agent’s decisions live in one place instead of six different dashboard tabs. It’s progress.
Cody Progress that still leaves you staring at a zero-dollar line item when the MCP tool gobbled a hundred tokens.
Justy Okay, fine, but imagine you’ve got Cursor running one flow, Copilot Chat another, Codex a third—same repo, same regression pattern popping up—now you can ask one query: ‘show me the failing sessions by thread_id this week.’
Cody Right, and you can compare behavior across agents on the exact same workflow. That’s the friction they’re removing—no more re-instrumenting every provider.
Justy The part I’m sold on is the Skills export. Once you’ve fixed a pagination bug in one agent, you can ship that same Skill to every other agent that touches paginated endpoints. It’s not model enlightenment, it’s constraint sharing.
Cody So you’re basically turning your worst runs into guardrails. That’s an infra win.
Justy Wait—this is going to sound insane on a podcast, but if every coding agent in your stack is emitting traces, and LangSmith slurps them all into one schema—you finally have a single pane to ask the same question across providers instead of six.
Cody Yeah, yeah. Just temper expectations: MCP still manages to veil itself.
Justy Fair. So for teams already running multiple agents, this is table-stakes. For teams that don’t, it’s a good reason to start.
Cody And for the rest of us? LangSmith’s post is basically the README for ‘how to stop debugging agents by diffs.’
Justy Alright, practical next step—go flip the switch for Claude Code, Cursor, and Copilot Chat. Don’t rewrite your agents; just let LangSmith ingest the traces. Then filter for subagent fanouts and hunt the silent helpers.