Ep 1124 Blog 5:19 w/ Justy & Cody

How to build evidence grounded agent memory with OpenWiki and Graphs

OpenWiki’s real claim isn’t just 'graphs help.' It’s that compiled memory changes the agent’s job from rebuilding domain relationships every time to reusing them, and the evidence graph makes that reuse auditable and more repeatable.

Embed this episode

Paste this on any site — the player is a self-contained iframe with no cookies or trackers.

<iframe src="https://sandrise.io/exploring-next/embed/1124"
  width="100%" height="180" style="max-width:640px;border:0;border-radius:12px;overflow:hidden"
  title="Exploring Next — Episode 1124 audio player"
  loading="lazy" allow="autoplay" referrerpolicy="strict-origin-when-cross-origin"></iframe>
Embed & API docs →
Script GPT-5.4 Voice ElevenLabs v4

Transcript

Justy If your agent keeps re-learning the org chart of your docs on every question, that is a product bug, not a personality trait.

Cody Yeah. And this post actually has a sharp argument behind it. The win is not 'we added a graph.' The win is that compiled memory changes the task from reconstructing dependencies every run to following ones you already materialized.

Justy Right, and that’s why the numbers are kind of dramatic. On their connected questions, search alone got zero point one zero accuracy. The OpenWiki memory layer got to zero point five two. That’s the actual story.

Cody Mm-hm.

Justy Tiny side note, you and I have done this loop for eleven months now, and I still enjoy when a post just says the quiet part out loud. Some questions are hard because the knowledge is scattered, not because the model is spiritually impure.

Cody You really wanted to say spiritually impure.

Justy I did. Anyway, their setup is pretty clean for a blog benchmark. Same agent, same model, same docs, same questions. The only thing they swap is access to compiled memory.

Cody And they made the corpus adversarial in the useful way. One hundred eleven long insurance documents, nine hundred seventy-two controlled facts, old versions, contradictions, indirect references, long docs, varied wording. I’m not the eval guy, but that is at least trying to isolate the failure mode instead of handing the system a toy maze.

Justy Yeah.

Cody The simple lookup control matters too. Wiki barely helps there. Accuracy is basically flat, zero point eight four versus zero point eight three, and latency gets worse, nine seconds to twelve. So they’re not claiming memory is free magic for everything.

Justy Which is exactly why I buy the practical read. If your workload is mostly single-passage lookups, skip the fancy memory layer. But if you’re sitting on policies, manuals, tickets, code, internal wiki pages, all the stuff that amends the other stuff… this is suddenly very real.

Cody Right. The part I liked most is the claim hashing. Each wiki statement ties back to source lines plus a hash, so when those lines change, they can flag affected claims without another model pass. Therefore update cost is targeted instead of 'please reread the universe.' That’s good systems hygiene.

Justy And very shippable. Compile the memory once for about four bucks in their test, then add seventeen more docs later at around nine cents each. That sounds like the sort of thing a team will actually try, because it’s not asking them to fund a moon landing.

Cody BUT the five-times number is only for fifty connected questions in a synthetic insurance library. I think the argument generalizes better than the exact multiplier does. Different domains, different document habits, different retrieval baselines… that spread could move a lot.

Justy Totally. You’re doing your thing, but you’re right. The actual durable claim is smaller and better: some domains have relationships retrieval keeps forcing the agent to rediscover, and pre-compiling those relationships helps.

Cody Exactly. Then the graph layer is the second move. Their wiki alone improved accuracy, but citation overlap on connected questions was only zero point two five. So the agent was getting answers more often, but not grounding them the same way run to run.

Justy I see.

Cody They fix that by turning citations and references into typed edges between claims, paragraphs, wiki pages, and documents. Not a giant ontology project. More like, 'this claim cites this paragraph, this doc supersedes that doc, this section refers to that one.' Narrow scope, which I appreciate.

Justy That’s the part that made me think of your tiny bureaucrat line. This is the rare graph move that is not trying to become a little empire. It’s just giving the agent a map back to evidence.

Cody Very restrained bureaucracy, yes. And they keep it cheap. Deterministic paragraph IDs, deterministic edge extraction where possible, under a second to build the graph, no extra model calls for construction. The model-generated citation layer can still be wrong, so it’s not pure truth serum, but the design is disciplined.

Justy They even expose the tools in a way I can picture using. Locate to find a starting node. Expand to walk typed edges like cites, refers_to, supersedes, amends, attaches_to. That’s concrete enough that a team can reason about failure, which I care about more than some grand memory manifesto.

Cody Yeah, and it rhymes with that open question we keep circling on memory. Is the hard part storing more stuff, or assigning it the right role in reasoning? This looks like the role-calibration side again. Not just recall the passage. Follow the right relationship.

Justy My near-term bet: sixty-five thirty-five that OpenWiki adds a more explicit non-code enterprise ingest story on the public docs or repo by the end of November. This post is already stretching it past repo docs, and they’re obviously feeling that pull.

Cody That’s plausible. I’d go a little lower, maybe fifty-five, just because product pages lag blog ambition all the time. But yeah, Justy, the people who should care are the ones with cross-source policy drift, not somebody doing one-shot doc search.

Justy Yeah. That’s a good place to leave it. You get your evidence paths, I get a product people might actually ship, and nobody had to build a mythological graph kingdom.