Scaling coding agents means designing for repository coordination

Running several coding agents at once sounds like a compute problem: add workers and let them write code. GitHub’s 5 November engineering post describes a less obvious limit. A busy shared repository can become a coordination problem, because every push creates more work for the writers and for the systems that read the new commit.

GitHub says its new storage architecture is designed for repositories receiving thousands of writes per second. In internal testing, it demonstrated 5,000 concurrent agents and 7,200 pushes per second to one repository. Those are company-reported test results, and the architecture was described as a limited private preview, with a broader rollout planned for the following year. The interesting part for most engineering teams is the shape of the design, not the headline throughput.

Immutable data and mutable pointers have different costs

A Git push stores objects—commits, trees and blobs—then updates references such as branches. Git objects are immutable and content-addressed: two writers can upload different objects without changing one another’s data. A branch reference is different. It is a small, shared pointer, and concurrent updates to the same pointer need an ordering rule.

GitHub’s design leans into that distinction. It stores pack data in Azure Blob Storage, keeps reference and pack-manifest state in a separate replicated service, and uses disposable virtual replicas to serve reads and writes. Background compaction runs on separate nodes. This decouples storage, request serving and maintenance so that each can scale on its own.

That architecture is specific to GitHub’s scale and guarantees. A typical product team does not need to copy it. But the principle—identify the small piece of state that truly needs coordination—applies to job queues, collaborative editing, workflow engines and agent orchestration.

More agents also mean more readers

One push often triggers CI, deployment checks, code review automation and other agents fetching the new branch. The write therefore amplifies into reads. If a team only measures agent execution time, it may miss the latency and cost added by repository fetches, repeated checks and contention on a shared branch.

Before scaling the number of concurrent agents, measure the full loop:

• How many commits and pushes does one task produce?
• How many CI jobs, fetches and review actions follow each push?
• Which branches or files do agents contend over?
• How often do they need the full repository rather than a small set of files?
• What is the time from completed work to an integrated, reviewed change?

These numbers reveal whether the bottleneck is model latency, repository coordination, test capacity or human review. Adding more agents helps only when the whole pipeline can absorb their output.

Practical patterns for a busy repository

Use isolated worktrees, branches or short-lived clones so agents do not overwrite one another’s working files. Keep task scopes narrow and tell each agent which files it owns. Treat commits as immutable outputs and make integration an explicit step with conflict checks and tests.

Reduce work triggered by every push. Batch related changes where practical, cancel superseded CI runs, and avoid repeatedly fetching large histories when a sparse or partial checkout will do. These optimizations reduce fan-out without weakening review.

Finally, make coordination visible. Track push rate, CI queue time, conflict rate, failed integrations and time waiting for human approval. A dashboard of agent counts alone can show more activity while the delivery system gets slower.

A useful way to scale

GitHub’s post is a reminder that agentic development changes the workload pattern: more concurrent writers, more frequent updates and much more downstream read traffic. The general engineering response is to separate immutable work products from shared mutable state, minimize the amount of state that must be ordered, and scale maintenance work independently from serving work.

Start with measurements in your own repositories. Increase agent concurrency gradually, watch the complete integration loop, and keep a clear owner for the final change. Fast parallel work is useful only when the repository can turn it into reliable software.

Sources: GitHub Engineering · GitHub