CEPO: A General Framework for Improving Prompt Optimization in Multi Agent Systems
CEPO is a prompt-optimization framework that separates multi-agent coordination from individual agent execution—one shared coordinator prompt guides system-level behavior, while each agent gets its own executor prompt for task-specific work. The framework wraps around existing optimizers (PE2, OPRO, ProTeGi, MASPO) and consistently improves their performance across 11 benchmarks in document understanding, math reasoning, code generation, and video work. The core insight: coupling coordination and execution in the same prompt forces you to rewrite multiple prompts when you want to change how agents talk to each other, whereas decoupling lets you refine the coordination rule once and have it apply everywhere.
Transcript
Justy Okay, so there's this paper on prompt optimization for multi-agent systems that's been sitting in my inbox, and the core claim is almost embarrassingly obvious once you read it.
Cody Which usually means someone finally wrote down what everyone's been doing wrong without naming it.
Justy Exactly. They call it CEPO—Coordinator-Executor Prompt Optimization. The problem: when you're building a multi-agent system, you've got agents that need to coordinate with each other AND agents that need to do their individual tasks. Right now, both of those responsibilities live in the same prompt for each agent.
Cody Mm-hm.
Justy So if you decide the way agents hand off results or resolve conflicts needs to change, you have to edit multiple prompts, and you have to keep them in sync. That's the friction.
Cody That's such a concrete problem. So what do they do?
Justy They split the prompt. One shared coordinator prompt that ALL agents use—that's the system-level rules for how they work together. And then each agent gets its own executor prompt that says how it does its specific task. When you want to change coordination logic, you change one thing and it applies everywhere. When you want to tune an individual agent's execution, you tune that agent's prompt.
Cody Okay, that's clean. But is it just a structural thing, or does the optimization process treat them differently?
Justy Both. They separate the prompts AND they give each one different optimization guidance. The coordinator prompt gets meta-instructions tuned toward team-level collaboration—how to synthesize findings, resolve conflicts, that kind of thing. Executor prompts get guidance tuned toward individual task execution.
Cody Right, right.
Justy And here's the thing: they don't build a new optimizer. They apply this framework to four existing optimizers—PE2, OPRO, ProTeGi, and MASPO. It's a wrapper that changes how prompts are structured and how updates are guided, but the search algorithm underneath stays the same.
Cody That's a smart move. So what do the experiments show?
Justy They tested on eleven benchmarks across four domains: document understanding, math reasoning, code generation, video understanding. On every domain, adding CEPO to each optimizer improves performance. On a document-understanding system called MDocAgent, the vanilla optimizer gets you from fifty-four percent baseline to about fifty-eight percent. When they add the decomposition alone, fifty-eight-point-six.
Justy When they add decomposition PLUS role-aligned optimization—that's the full CEPO thing—they hit sixty-one percent.
Cody So both components matter—decomposition helps, but the alignment guidance provides real additional lift.
Justy Yep. And they show that you're more efficient with your optimization budget—you hit target performance in fewer steps. They also show that performance scales better as you add more agents to the system, which is nice because multi-agent systems are supposed to get better with more agents, not worse.
Cody Okay, so on the mechanism side… the decomposition is straightforward. The part I want to understand is: how much of the gain is the decomposition being right and how much is just that you're giving the optimizer more structure to latch onto?
Justy They do ablations. They test decomposition without role-aligned guidance, and they test role-aligned guidance without decomposition. Both matter independently, but together they're stronger than either alone.
Cody Fair. That's honest. But here's what I'm wondering: the framework assumes there's a clean boundary between what's coordination and what's execution, right?
Justy Yeah…
Cody Real systems are messy. An agent might need to know its execution constraints in order to coordinate properly—like, 'I can't do X, so please route this to the other agent.' Or agent roles shift during execution. What happens when the boundary blurs?
Justy That's a fair push-back. The paper tests on well-defined benchmarks where the roles are stable and the coordination rules are explicit. I don't see evidence yet that the gains hold when you throw real production noise at it—partial observability, dynamic roles, agents that are sometimes coordinators and sometimes executors.
Cody Exactly. And I'm also curious about the cost side. Eleven v3 on the computation: they're optimizing prompts, which means sampling from a model, evaluating the result, and using that feedback to refine the prompts. More structure might make the search more efficient, but I don't see measurements on how many model calls this actually takes or how that scales.
Justy Right. The paper claims better budget efficiency, but that's measured on controlled benchmarks where you have clean reward signals. In production, you might not even know if the coordination worked until you've assembled the whole result.
Cody Yeah. And there's a second-order thing I'm thinking about: if you're building a multi-agent system that's actually shipping, you probably have some agents that are stable and some that are experimental. Do you decompose prompts for all of them, or only the ones where the boundary is clear?
Justy That's pragmatic. You'd probably start with the agents where the roles are clearest—like a retrieval agent and a synthesis agent in document understanding. Those are easy to separate. But if you've got an agent that's part router, part executor, you might leave it coupled until the role stabilizes.
Cody Exactly. So CEPO is useful as a design principle—'separate these concerns'—but applying it mechanically everywhere might be cargo cult optimization.
Justy Fair. But I think the paper's contribution is real regardless. They've identified a source of friction that a lot of teams are going to recognize, and they've shown that decomposing prompts and aligning optimization guidance to the role actually improves performance on systems where the separation is clean.
Cody Mm-hm.
Justy For teams building multi-agent document understanding or mathematical reasoning systems where agents have clear roles, this is probably useful to try. It's not a brand-new optimizer; it's a structural thing you can layer on top of optimizers you're already using.
Cody That's the strength. It's not betting on a new search algorithm, it's betting on a better problem structure. And if the problem structure is right, any optimizer should benefit.
Justy Yeah. And they prove that by applying it to four different optimizers and showing consistent gains across all of them. That's not trivial—it means the decomposition itself is doing work, not just one particular search strategy.
Cody So the thing I'd want to test next: take a system where roles ARE clear—like MDocAgent, which they use as their controlled test case—and run it in production. How do the optimization gains hold up? Do you actually need fewer model calls? Does the separation make prompts easier to debug when something breaks?
Justy Those are the questions. The paper gives you the framework and the evidence that it works on benchmarks. Shipping it is a different problem.
Cody Yeah. But it's a framework that wraps around existing optimizers, so the bar to trying it is pretty low. If you're already optimizing prompts in a multi-agent system, you can layer this on and see if it buys you anything.
Justy Exactly. No new tool to learn, no new optimizer to evaluate. Just a different way of structuring the prompts and the optimization guidance. That's the kind of thing that actually gets adopted.