Posted by Philpax 5 hours ago
For example, you can imagine, at the extreme, that agents could get caught in code change loops: a code change seems appropriate, it is implemented, it turns out there's some external factor which makes it not appropriate, the change is reverted, then that same code change seems appropriate again, etc.
Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
Then again, my standard instruction is something like "{bug description}. Find the root cause and propose possible solutions" so I guess I fit your profile
Can you even do the latter without the former?
This is a great idea, now they just have to use/do it. Maybe they can use it effectively by cloning the repo and having all the history at hand, rather than continuously scraping web pages with git content formatted for human brwosing.
This is true, but at 30,000 merges a day we're talking about nearly 1 million merges a month, and purely autonomous agentic merges tend to be noisy. Is this still a tractable problem, even for an agent? Do revisions older than a few days, other than perhaps release snapshots, have any relevance for such a process?
Of course that does require some folder structure, and agents are not necessarily great at that, but they're still decent enough that you do end up with many folders not being impacted hour or even daily.
1. they have no-cap token spend allowance internally
2. their agent swarms are amongst the largest if not the largest, again because #1 above.
3. their git repo is enormously large (100GB uncompressed git history on main, 25GB compressed).
4. their merges per day count grew an order of magnitude from May to September to 30000 times per day, they are not seeing it slowing down anytime and probably only going to accelerate.
5. all of their code agents are networked and has network access, full local pull down is not needed at all and too slow (see #3)
6. their design of the Scion is based off prior art Google’s Piper and Meta’s Sapling.
Given their own constraints and characteristics of their own. It only makes sense to build their own homegrown solution based on prior arts. But this was NOT a complete redesign from scratch - so it's an full online download only what you need type of source control system. The problem is the version control software itself but its prior tradeoffs are misaligned with their scaling needs.
PS, if anyone noticed, Anthropic starts to concealing their employee full names and replaced with only first name + last name initiate, not sure why.
PPS, the project Scion is so bad, scn is worst, it reminds people of prior SVN, which is bad by itself.
I might be completely out of context, but why is it only 4x compression? Do they commit a lot of binaries? I just don't understand why would code compress so badly.
I mean 30k CLs per day isn't much. That's one every few seconds. Well within what a decent perforce server can do.
Gives me a vaguely creepy feeling, but it's reasonable.
For how the Linux Kernel, a highly decentralized, completely open source project, with a single monotonic product, tyrannically lead (a good thing) - is developed.
Permissions, Partial Checkouts, Sub Repos, Vendoring, pub/sub... How many companies bought into git and then had to buy more tooling (like artifactory) because their 3rd party dependency's needed a home. Thats a feature that git does not support because git, as designed, did not need it.
Think about how much better your life would be if every time you typed npm,pip,goget,cargo it fetched THROUGH your repo. If those updates triggered code reviews of whatever changes you're going to consume. You are code reviewing those updates aren't you (someone is, cause supply chain attacks don't last that long)
Git has some weak points, Google doesn't use it for a reason. Epic created lore for a reason, This exists for a reason. Im not sure that any of them are the next thing that replaces GIT but it's getting to be time. Tools like this are going to keep popping up till someone gets it right.
I know you are joking, but the answer is the same. Because the agents need it, not because any human need it. The Agents quite actively use the history. It's more effective than any comments or separate documentation might be for the agents working on the code.
Do you think each and every one of those are going to be "good" commits, commits that still "compile" or function. Possible, but unlikely given the insane velocity.
Give it six months and there'll be have Optimus robots toggling switches to load terrabytes of binary programs directly into memory. Nobody wrote it, nobody will read it, but it will run everything.
It works, but there is a significant amount of churn. I find when the swarms use git it's less churn and less token usage.
That is, the deterministic part is the platform that is being built and that non-deterministic agents will use and build it, but that layer is a key part of how they do work and will continue to be.
The big baddies of build have worked this way for a long time already. Git surfaces a single Merkle tree id for the whole repo on each commit, but build tools like bazel/blaze, buck, pants, et al look at hashes of subtrees and use cached artifacts for the parts of the tree that didn’t change.
Your CI no longer becomes a binary signal on the code and more of a continuous signal on how much your “this works” pointer has caught up with your “this is the latest code” pointer.
All the others seem necessary. Maybe you don't need branches at all, of course we do! Things have to have names.
There could be some fun cryptographic ways you could do it directly inside of git.
Pulling that off successfully at scale would be pretty incredible to watch.
We built all this tooling because it's useful. If you replace the human with an agent the tooling doesn't stop being useful. If anything it becomes more useful, since AI Agents excell at reading tool output but struggle with maintaining long-term state
I believe you're mostly thinking of version history though. There’s plenty of other reasons to have source control we still need, like guarding against racing writes or continuous integration to ensure correctness. Even some industries require version history for compliance reasons.
I do love how AI is challenging us to rethink our tools though rather than blindly reaching to git for every project.
In fact most of the architecture work I’m building, assumes that there will not be a human reading code, they read (effectively) UML and data diagrams
so most of the interesting parts I’m working on now are how do you make code blazingly fast, formally verified and memory safe if you remove the constraint of human interpretability of machine instruction
From the post (especially the end) it seems that they are essentially letting software be fluid.
This is how we solve cybersecurity, by making code into constant shifting sand, so a vulnerability never truly lives long enough to be exploited at scale. Sure, a hacker may occasionally pick out a vulnerability like a bear picks salmon out of a river, but it makes larger scale cyberattacks impractical. The code is a firehose, constantly in flux.
So "no direct human supervision" seems like a misunderstanding. Article even bigs up the purpose of engineers with a hilarious (to me at least) line about auto adversial reviews resulting in over engineered designs.
These are just some of the things - there may be many more.
The first thing that hits me is, did you consider not going all in on a monorepo? I know it has been the trend in general, but it seems a bit of a red flag when you go from "let's have a monorepo it will make merging easier" to "let's redesign version control because merging is too hard now" without the middle question "should we really have a monorepo?"
As a derivative thought I'm curious if AI tooling is tipping people more towards monorepos now since it gives the agent the whole context of the app in one go? Or is this independent - I can easily check out 5 repos in one directory and run an agent there.
The guy who ran that project called it his "$LASTNAME law" that every large company will at some point write their own source control system.
I laughed.
About a year later I went to Google, and within a couple of years Google also did the same thing.
Not at all surprised. Git was just good enough (and cheap enough) the companies didn't need to do this to escape the {phsyical scaling, cost scaling} problems that most commercial SCM systems had.
Cursor scaled git to way more writes/s than Anthropic did with their rewrite.
> ..., the latent demand for merging code is practically infinite, and every bottleneck we remove immediately leads to a further increase in the amount of code we produce.
If the goal is to write software which solves real problems, how can the demand for code grow infinitely?
Inventing new problems caused by our "solutions"
> What is the code doing
Don't you know? We're just 6 months away from AGI; no one will ever need to read source code ever again!
> What problems are being solved?
How do we build a house of cards that's so tall, people value it at trillions of dollars, and build it so quickly, people don't realize it's worth far, far less?
> goal is to write software which solves real problems
No, no, you silly goose. The goal of software is to make gluttonous amounts of money. The side effect just happens to sometimes, occasionally solve real problems
If you are constantly churning your codebase to the point that you need to invent a system that handles your churn... then you must at least be admitting that the previous 6 refactors you wrote were not good, but NOW you have it right.
I feel that good, well designed code change by accretion, not by rewrites. Code that remains stable for a long period of time was likely better than code that needs be continuously rewritten (at least with humans writers).
But maybe this is just a human-centric view of what good code means? That LLM "slop it until it works" approach to development will have a different measure of what quality looks like?
i really don't want to brick my camera, so the deterministic emulator stack lets me do integration tests end-to-end. if i was doing this by hand with no ai, i'd probably have to run stuff on the camera just to test it. but because ai, i get deterministic simulation testing, at the cost of needing to burn $100 of subscription on it.
ai code is more like clay you gotta continuously keep shaping over time, but you can also get very very far, very quickly if you decide the code itself doesn't matter and all you care about is results.
It seems to me like the AI labs are just pushing the envelope with blind faith that”something” will happen if they just get enough development speed. Nobody seems to have a coherent idea of what that something is.
I took a photo of a tree with chatgpt recently and asked it what species it was. That was kinda neat I guess. Someone forked a ps4 emulator with a bunch of Bloodborne specific improvements so its more playable on my setup. It's kind of okay for medical stuff.
All that busy work was for going from zero revenue to $50B in revenue in about 3 years, and releasing AI systems that are getting better and better at a ridiculous rate. AI isn't solving all the world's problems but you can't deny that they are extremely successful with this method of operating.
I definitely think that's an interesting question though. I know that in the physical product development world I'm seeing really odd asymmetric speed ups. Creating simulations is essentially automated at this point, stuff that would take days to code by hand can be done in an hour. Firmware debugging feels 3x faster than before. But the big slow stuff like manufacturing, assembly, and testing are completely unchanged. All of that little stuff probably adds up to maybe a 5% faster product development cycle than before if you're really leaning into AI. It's the type of thing that will compound over time especially if it's happening all over the economy, but won't blow your socks off.
"A world where starting is incredibly easy and finishing is more complicated than ever"
LLM's do allow you to oneshot huge starting bits of work in record short time. The frontier of the amount of work they can oneshot is clearly getting larger and larger. But I am still not convinced that they allow you to maintain existing work at a cost that is below a human being.
But, if the frontier of what you can oneshot continues to grow unbounded, then it really will not matter. We will eventually oneshot all software and maintenance will be a thing of the past.
... But that is a very big bet.
Yeah I wish this whole energy was towards optimization and improving hardware.
One thing that people are finding hard to believe is that the frontier labs are cultists who want to "create god". Hence the focus on improving intelligence so they can get to recursive AI, as well as the fear-mongering.
Aanthropic-openAI duo cannot go bankrupt soon enough. This industry is going to flourish with those two out of the picture. They're the pets.com of 1999 trying to have a say how the internet should be.
Imagine that you have git but you don't even have a concept of programming language more complex than assembly. All your commits and git history are pure assembly. Everything is cool, until one day C++20 randomly spawns into existence, and a bunch of guys decide to use it in the company. The issue is, the compiler generates so much assembly, that your existing workflow cannot keep up with it. Your assembly git repo grows into hundreds of gigabytes, with PRs that are pretty much complete rewrites coming every single day and there's no way to review any of that shit.
What Anthropic is doing is redoing git from scratch so that it can handle all that assembly, while the correct thing to do is to commit C++ code into the repo, with deep trust that you'll never hit a bug in the compiler - basically impossible for initial versions of compilers, taken for granted nowadays.
What I'm saying is, Anthropic should create a versioning system for prompts, because they are the true "source code". Given the same prompt and the same sequence of random numbers, LLM will generate exactly the same code, which sounds awfully close to how compilers work. Once in a while put the generated code as a point of reference+ caching and presto.
The good ones do.
Citadel Securities, Renaissance, Jane Street, HRT, Jump, consistently do 50%+ annual returns over the past 15-20 years. They obviously don’t accept outside investor money.
Now, hedge funds that accept outside money like DE Shaw, Bridgewater and shit, yeah, they barely outperform S&P500.
Because fees as a manager of someone else’s money can be higher than the full amount of the returns if it were just your own money.
Fees on someone else’s capital: 1% of 20% of 1bn = 2m
Returns on your own capital: 20% of 1m = 200k
Anthropic could easily build a CRM equivalent to or better than salesforce but they still wouldn’t be able to compete with salesforce. It makes a lot more sense to sell salesforce tokens instead.
Imagine how much money they'd raise if they just said one day "we are going after salesforce bottom line now." It would be like chumming the water.
As many in the industry say now - writing code was never the hard part
Maybe that's their way of becoming Google?
What do you think Claude (chatbot web interface) is?
My only explanation is that Anthropic's search backend sucks.
However, I can tell you someone who fits all those criteria: Levi Strauss. Strauss founded a company to sell dry goods, as well as tents and jeans, and eventually specialized in those denim jeans. Levi Strauss became a household name and an enduring success of the Gold Rush.
Therefore, I say that this motto is flawed; so let's correct it: "In a gold rush, sell the work clothes!"
Fielding a web index takes a massive amount of compute and storage of a very different kind than what Anthropic is using. AI is extremely heavy on GPU, maybe somewhat on storage but (comparatively) light on CPU. They need CPU to train coding agents, obviously, but scaling to a billion user web search is a whole different thing.
But it's a sad example, for open-source and for small engineering companies. Instead of infrastructure being more easily shared with the collective, or valued enough to build a business, now companies can tailor even major infrastructure to their own needs, and avoid all that coordination.
I'm glad they publish their notes to encourage others, but I suspect we're entering an epoch where confidentiality matters most.
Anthropic's problem was merging code at scale, but the scale problem I have with SCM is really merging all the related docs; these get lost in a stream of a forest of comments, PR's, and documentation, all of which (like a wiki) goes rapidly out of date before anyone notices because none of it is automatically correlated.
Large companies have always done this. Tons of early 2010s open source projects came out of reimplementing white papers from closed-source Google projects.
It’s also the major downside of this kind of thing. Once someone gets acclimated to how things are done at a large company with custom tools, it’s not always easy to switch to another process, especially a simpler one. There are tons of stories about FAANG engineers joining a startup and being useless.
There’s real value in having consistent tools across an industry. It makes hiring and training easier. Even if you’re convinced that a LLM could crank out a payroll or ordering system that’s perfect for your org, you still have to train the accountants and admin assistants to use it. And you’ve now become the customer support.
The bar is lower with version control, but it’s still more friction than git + GitHub + a well-known workflow.
This is the weirdest part, because you know, coding is solved, so anyone could create their own one, so why not publish it?
You think you have it bad now when GitHub goes down, wait until you're stuck with an online only client.
If I were still manually merging conflicts the p4merge tool was GOAT
I used it in git
I feel like the advantage of having these models in house can't be understated. Companies with access to inferior models, trying to compete, will be left in the dust when it comes to general development.
Just a reminder to simply embrace their superior capabilities instead of feeling threatened.