Inside GitHub's new Git storage architecture
GitHub’s Project Rubicon argues that agent-scale Git is less about making every write globally coordinated and more about isolating the tiny piece that truly needs ordering: branch and tag references. Edmund sees a practical platform upgrade for busy repositories; Geffen likes the mechanism but notes GitHub has shared internal stress-test results, not production latency or failure data yet.
Transcript
Edmund GitHub says it can now take seven thousand two hundred pushes a second. On ONE repository. And the way they got there is kind of sneaky. They decided most of a push doesn't need to wait its turn.
Geffen Which is the real claim here. Git objects are immutable. Once an object exists it never changes, and its content determines its identifier.
Edmund So two agents can upload different objects at the same time without ever negotiating.
Geffen Right. The only mutable bit is the branch reference, the tiny pointer that says which commit main points at.
Edmund And Rubicon, the new storage architecture, separates the bulky object data from that contested pointer update.
Geffen Which is a much cleaner scaling target than treating every push like one giant transaction.
Edmund What I like is it's an under-the-floorboard change. You still push, fetch, branch, your protections stay. But if you're running a very busy repo, Git infrastructure stops being the thing slowing all your agents down.
Geffen And GitHub is explicit about the pressure. Their busiest repositories today see at most dozens of writes in a second. Usually bursts.
Edmund Dozens. That's it?
Geffen That's it. Their expectation is that parallel agents turn those bursts into sustained traffic. And every new commit triggers C I pulls, deployments, other agents fetching context.
Edmund The read fan-out.
Geffen Which is why this isn't just a write benchmark. They showed one repository's daily requests growing roughly fourfold, March through May. So the response is to target two orders of magnitude more scale, not tune for the next small bump.
Edmund The architecture's pretty legible, too. Spokes still handles routing, tenant placement, fairness. Under that it sends requests to virtual replicas. Disposable compute nodes whose local disks are caches, not the source of truth.
Geffen So when one disappears, another keeps serving, and the replacement rebuilds its cache as traffic arrives. Repository gets hot? Add replicas.
Edmund Very different story from nursing one giant permanent Git server through every spike.
Geffen The durable split matters more, to me. Packs, indexes, read accelerators live in Azure Blob Storage. References and the pack manifest live in Ref-CAS, GitHub's replicated log service.
Edmund Compare-and-swap being… remind me.
Geffen A reference update only succeeds if the state is still what the writer expected. So on a write, a virtual replica indexes the incoming pack, uploads it, then commits a Ref-CAS transaction. That adds the pack to the manifest and updates the references.
Edmund And reads?
Geffen Check the reference version, fetch the needed packs into cache, serve locally. Compaction happens elsewhere, merging packs, removing unreachable objects, without stealing request-serving compute.
Edmund That last bit's the product consequence for me. Maintenance work shouldn't show up as random sluggishness right when a repository is busiest. Nobody opening a pull request wants to learn about pack compaction because their agent fleet woke up.
Geffen No, that would be a bad morning.
Edmund And honestly, I think the technical argument holds. They identified the actual coordination boundary instead of pretending all Git data has the same consistency needs.
Geffen Hm.
Edmund It fits that agent-scale thing we keep circling, right? Capacity isn't only model quality. It's whether the surrounding system can absorb lots of small, concurrent actions.
Geffen I'm with you, with one limit. Ref-CAS can serialize reference changes efficiently. It cannot erase genuine logical conflicts.
Edmund Meaning?
Geffen Meaning five agents all try to advance the same branch from incompatible histories, somebody still loses or retries. This improves infrastructure contention, NOT collaboration judgment.
Edmund Yeah… okay.
Geffen And the strongest number is an internal test. Five thousand concurrent agents, those seven thousand two hundred pushes a second, reads staying consistently low latency. Meaningful engineering evidence. But no latency distribution, no production workload, no failure-recovery measurements. It demonstrates headroom, not a finished universal verdict.
Edmund So who cares right now… probably teams with a monster repository, heavy continuous integration, lots of automated writers. For most repositories it should be invisible.
Geffen Invisible being the win, when you're rebuilding a core service underneath an existing workflow.
Edmund Kind of, yeah.
Geffen It also lands in the live infrastructure-first versus capability-first fight. GitHub's betting agent adoption makes repository traffic explode, so coordination and storage design become the constraint. That doesn't settle whether everyone reaches that scale. But it's a serious entrant on that side of the debate.
Edmund Rubicon, though. Keep the name dramatic. I'm mostly glad somebody moved the bottleneck before it broke.