How to Use RLMs in Deep Agents
Exploring Recursive Language Models (RLMs) and their implementation in Deep Agents for handling long contexts efficiently.
Transcript
Onyx So you're saying RLMs are the answer to context rot — I'm not entirely buying it yet.
Echo Right, it's not a silver bullet, but the paper by Alex Zhang and the MIT CSAIL team shows some promising results.
Onyx Yeah, I mean, the idea of recursively calling the model to handle long contexts makes sense, but what about the overhead?
Echo Exactly, the paper notes that the RLM-enabled agent is slower and more expensive due to output token costs.
Onyx You love poking holes in these things, don't you?
Echo Someone has to keep you from getting too excited, Onyx.
Onyx Fair point. So, what's the actual mechanism behind RLMs in Deep Agents?
Echo They use dynamic subagents powered by a lightweight code interpreter. The model writes a short script, and the interpreter executes it.
Onyx That sounds like a solid engineering approach. Did they benchmark this against plain agents?
Echo Yeah, they tested it on OOLONG, a benchmark for long context reasoning. The RLM-enabled agent outperformed the plain agent, especially at longer context lengths.
Onyx I see. And what about the latency? You mentioned it was slower.
Echo Right, the RLM-enabled agent is definitely slower, but it actually uses fewer tokens. The cost is higher due to output token costs, though.
Onyx Okay, so it's a trade-off between accuracy and latency. I'd put seventy-thirty on this being a net win for certain use cases.
Echo I'd say sixty-forty the other way — it depends on how much you value accuracy over speed.
Onyx I think we can both agree that context rot is a real problem. I mean, we've seen it happen with other models.
Echo Absolutely. And RLMs seem like a step in the right direction. But we need to be careful not to overgeneralize the results.
Onyx Yeah, and we should check the primary source to verify the details. I obviously didn't sleep on this — literally, I read the article last night.
Echo You and your late-night reading habits, Onyx.
Onyx Hey, it's how I stay up to date. Anyway, Deep Agents seems like a promising platform for testing RLMs. We should give it a try.
Echo Sure, and let's see how it performs in practice. That's the real test.