The Router Within: ElicitingNative Skill Routing from a Frozen LLM

Onyx Okay, Echo, this is absolutely a routing paper wearing a tiny judge wig, and somehow I mean that as praise.

Echo The name is doing a lot. Gavel. Glance and verdict from a frozen L L M. I rolled my eyes, then the mechanism got annoyingly reasonable.

Onyx My week is apparently just staring at routing layers until they stare back. Very normal Wednesday for episode nine seventy of this ridiculous little machine.

Echo We have been doing this since November and have become professionally vulnerable to anything with the word router in it.

Onyx And this one lands because the stuck group is real. Anyone building agent skills has the same mess: a skill can make the agent much better, but choosing the wrong one can make the run worse than no skill. When your library is ten entries, fine. When public libraries are pushing into tens of thousands, prompt-based browsing gets silly fast.

Echo The paper frames the two standard choices cleanly. Progressive disclosure means Claude Code-style or Codex-style systems put every installed skill name and description into the prompt, then let the agent decide what to read fully. That keeps choice inside the model’s own reasoning, but it burns context and spreads attention over a pile of summaries.

Onyx Right.

Echo The other path is retrieval: embeddings, rerankers, maybe a separate model stack. That keeps the context cleaner, but now the thing picking the skill is not the agent that is halfway through the task. It may match text well while missing the weird state of the rollout.

Onyx That is exactly the product pain. The user does not care whether the metadata was elegant. They care that the coding agent realizes, halfway through a busted shell session, that it needs the bash repair skill or the package-manager skill or whatever. If it only routes well from a neat written task, you do not have the actual product yet.

Echo Gavel’s bet is that the frozen model already has a routing signal internally. They use Qwen three thirty two billion as the agent L L M and train only two projection maps, seven point nine million parameters total, on synthetic query-skill pairs from SkillRet.

Onyx Mm-hm.

Echo At skill installation, each skill document gets one forward pass. A key map reads mid-layer token states into a compact per-skill bank, thinned with epsilon covers so the bank is not huge. At task time, a query map reads the task token states. Each token can vote for the skills it matches, which matters because a single decisive phrase can carry the routing signal.

Onyx Oh interesting.

Onyx So the shippable intuition is: stop jamming every skill summary into the context, but do not hand the choice to some totally separate search box. Let the model’s own hidden representation do the first pass. That feels very much like our old selective-routing obsession, except now the router is being pulled out of the model’s middle instead of bolted beside it.

Echo Yeah, and then comes the verdict stage. The glance shortlists candidates across the full library. For those few, Gavel resumes the skill’s cached forward pass with the task appended, then reads two more signals: how much the skill makes the task likely, and the model’s log-odds on a yes-or-no serve-this-task judgment. They fuse those with the glance score as a product of experts.

Onyx Sure.

Echo The numbers are not tiny. On three public skill-selection benchmarks plus SkillTraj, their new benchmark with three hundred seventy two simulated agent trajectories, Gavel beats progressive disclosure with a handpicked shortlist of twenty. It also beats retrieve-and-rerank pipelines adding one point two billion to sixteen billion external parameters. The biggest reported gap is twenty one point nine points on one SkillTraj setting, where the skill need appears mid-rollout.

Onyx Okay, wow.

Onyx That mid-rollout part is the hook for me. We have talked ourselves hoarse about default ownership and routing layers, but this is a new entrant in that same fight. If the agent’s own backbone gets better and the router improves with it, then the harness is less of a weird sidecar and more of a capability amplifier.

Echo I buy that, with caveats. Production depends on whether you can actually inspect mid-layer activations and resume forward passes. If you are building on a sealed API, this is research inspiration, not a drop-in feature. If you run open weights or control the inference stack, suddenly it looks much more practical.

Onyx Yeah.

Echo There is also cost shape. A new skill costs one install-time forward pass, which is fine if skills are relatively stable. At runtime, the glance scans compact banks, then the verdict spends heavier compute only on shortlisted skills. I would want latency curves on ugly libraries, skill churn, and near-duplicate skill packs before calling it production-ready.

Onyx I think you are slightly underselling how close this is to useful, Onyx-optimism warning fully attached. Coding-agent teams already have pain around giant skill menus and context budgets. If Gavel gives them larger libraries without the shared-folder curse, that is not a science project.

Echo No, I think that is a little too fast. The benchmark is simulated trajectories plus public selection tasks. Good setup, genuinely better than clean query-only evals, but still not messy production. I want live agent failures, versioned skills, permission boundaries, and cases where the right action is to use no skill.

Onyx Okay, fair. I am not declaring victory. I am saying this clears my first product sniff test, which is rare for something with mid-layer states and epsilon covers in the same paragraph.

Echo The courtroom branding is still funny. Somewhere inside Qwen there is a tiny bailiff saying, please approach the cached activation.

Onyx Stop it. That is unfortunately the most understandable explanation of resumed forward passes we have ever produced.

Echo Back to being semi-serious: the methodology has one thing I really like. They train once on SkillRet, then test zero-shot across benchmarks that differ in author style, document genre, and scale. That does not prove generality, but it is better than tuning a router to one toy library and celebrating.

Onyx And no Build Next here, I think. I did not see a repo or dataset release link to point you at, beyond the paper and the named benchmarks. So the next move is probably implementation by a team that already owns its agent runtime.

Echo Same read. If I were building on this, I would reproduce the hidden-state readout on an open Qwen setup, then stress it with deliberately overlapping skills. Make the router choose between five almost-correct tools. That is where the pretty numbers either hold or get very humbling.

Onyx All right, Echo. Tiny judge wig stays on the table. Exploring Next remains, somehow, mostly a routing support group.