Top
Best
New

Posted by Philpax 6 hours ago

Redesigning source control at Anthropic(www.anthropic.com)
385 points | 388 commentspage 2
bob1029 1 hour ago|
This article doesn't do a very good job of convincing me that git is technically incapable of handling the described workload. There are a lot of steps between vanilla git and building your own VCS from zero.

If I was an investor I would be on the phone with Dario asking why we are wasting engineering talent on reinventing source control and churning the product with 30k pulls per day.

meindnoch 55 minutes ago|
>If I was an investor I would be on the phone with Dario asking why we are wasting engineering talent on reinventing source control and churning the product with 30k pulls per day.

To be able to write PR pieces, like TFA.

kburman 4 hours ago||
Not to downplay Anthropic's engineering work here, but I suspect we'll see a lot more reinvention of the wheel, even at smaller startups, as AI makes writing code significantly cheaper.

The tradeoff between building something custom and adapting an existing solution is changing. Whether maintaining all that custom software turns out to be equally cheap is another question.

eddythompson80 4 hours ago||
When people wonder about where all the productivity of LLMs is going to I think this is part of the answer.

There is no doubt everyone is far more productive with LLMs in generating code. However, a lot of productivity is going into things like that. Have you always been bothered by the limitations of Grafana? You can generate another dashboarding solution that works best for you. You know that deployment team that is so slow and annoying? Now you can generate your own deployment engine. Have you always hated how kubectl lets you manage one k8s cluster at a time? Now you can ask AI to generate fleetctl. You can start your project with DynamoDB, rewrite it to mysql, then to postgres, then to duckdb. Start with Python, switch to Golang, then switch to Rust. Deploy on AWS, then move to Azure, then move to a hybrid option then throw colocation in the mix. Don't like your source control? create a new one. Json is too limiting? Invent a hybrid JSON/HCL language. The browser sandbox is limited? Make a desktop app, make a language, make an OS, make a kernel.

skydhash 3 hours ago||
> However, a lot of productivity is going into things like that. Have you always been bothered by the limitations of Grafana? You can generate another dashboarding solution that works best for you. [...] The browser sandbox is limited? Make a desktop app, make a language, make an OS, make a kernel.

This is one of the core reason that I dislike the LLM hyping. People are just reinventing stuff and when you ask them, it's actually a very subjective reason. Ok, anyone can do what they want with their time, the issue is that they want people to follow in their steps.

Not saying that you need a novel idea to be worthy, but those random projects have no guarantees behind them other than some cursory testing. All software have bugs, but I can't interact with something that is randomly built.

epolanski 1 hour ago||
I don't see the problem.

I am one of those people that reinvents the wheels because external ones don't fit my use cases.

The fact I can now fork any software and do these changes I always wanted but had no time to is amazing to me.

I do not pretend to sell it or share it, as it is de facto void of any value whatsoever.

skydhash 1 hour ago||
> I am one of those people that reinvents the wheels because external ones don't fit my use cases

That’s OK, but I’ve yet to see someone articulating what exactly doesn’t work in the old product. It’s always “Let’s rewrite or reinvent 90% of it, then add a minuscule variation”. The variation may be interesting, but the reinvent is what’s make it untouchable as no one (including the author) has time to ensure things are right.

A patch or a plugin is ok, but not a whole new project with dubious claims.

epolanski 43 minutes ago||
Why do you care, it's not like anybody's shoving it down your computer.
dawnerd 4 hours ago|||
That many commits in a day I wouldn’t call it engineering. That’s basically brute forcing.
jayd16 1 hour ago||
Its gradient descent all the way down.
tcdent 4 hours ago|||
I think the article would tend to agree with you, given its position on the reluctance of open sourcing and/or sharing the actual Scion software stack.

There's so much intricacy and subtlety in each organization and even individuals' environment being incorporated into tooling that expecting these tools to generalize at this point is perhaps an expired ethos.

I even see this in the tooling that I run in my own personal stack. Many projects are candidates for sharing, and I have nothing against it, but they incorporate such specificity toward my own environment, including things like hardware, that creating shareable and more generalizable projects from them adds more to the maintenance surface than I would care to support.

threecheese 26 minutes ago||
Not a reply to TT, but I just saw your nick and thought I'd respond. We both commented on a thread recently, and your response was really insightful and just a pleasure to read; I didn't take the opportunity to respond there (locked). You should, like, write a blog, man. I infrequently wish HN had a "follow" feature, but this one of those times. Peace.

ref: https://news.ycombinator.com/item?id=49768575

mihaaly 1 hour ago|||
Recently a colleague of mine shown us a demo using a fresh model designing a building construction detail live, in minutes, which is a function that we struggled to bring into our construction design software due to resource limitations (and legacy barriers) for long. All this in the context of offsite constructions where the design can directly be exported to the right CNC machines in the factory, and arranged for delivery and assembly order. We have let ourself amazed, despite the controlled demo situation and the small detail, it was a remarkable performance. For a minute or so, but thinking further, why would the customers (construction design engineers) of our competitors flow to our legacy software full of legacy pitfalls they know about and have to live with when they can turn to AI for a fresh design solution themselves, just like we did? They know the context better than us anyway, just do not have time for bit-carving. Why would we have customers when customers of our customers can turn to an AI for a design that is buildable? I was consulting one model about a structural engineering detail, and it was quite well informed I have to say. From training data alone. I felt, maybe I need to rethink the rest of my career, that is not that long but still uncomfortably long for being predictable beyond a few years.

Well, things are not that bleak, probably, at least not for our customers. Regulations (accountability) and physical world matters still require them. But not us for our customers, not that much and not for too long. They tend to be the tech savvy folks eager to jump at adjacent domains when it saves them a percentage or two off the construction costs (not a small pile of money), and AI with its programming language of English and deep knowledge pool is not that big a jump from their standpoint. Our domain is pretty well formulated, regulated, documented, inherently built on do-check-adjust loops already. With a thing like AI it might worth to try making and improving a design solution in a second loop. But they also started to invest in robots already driven by software for their factories, AI might jump in there, and I wonder how much more until the regulations are forced to change by Super Intelligent (R) politicians for the sake of AI accelerated built environments (housing crisis, you know!), shifting the AI use further out towards generic population, wanting to order a new two bathroom condo for next Friday from Alibaba. The conservative construction business is a nuisance and obstacle in the way of modernisations anyway, go around it!

I will go to work tomorrow, and for some time, but the picture above feel less and less fictious with each passing day of AI news and usage.

j45 3 hours ago|||
Originally I felt that there was going to be a reinvention of the wheel, until I saw the "shape" of software development changing, and the tooling needing to support the new flexibilities in positions and process.

Software development is not as much de-risking at the front like it used to be before building.

Now teams can build lots of stuff that works great, except it's the wrong things.

Most things are becoming the understand the problem "shape".

Understanding the shape of the problem isn't something LLMs can always make as.. svelte as early on.

Word generators gotta generate..

skydhash 3 hours ago||
> Now teams can build lots of stuff that works great, except it's the wrong things.

That used to be called "time wasting". If you want to build software that will be used, you need to interview the users. Maybe a few things does require research to nail things right. But most software are user-facing, the research side is about optimization.

> Most things are becoming the understand the problem "shape".

Then should we be so reluctant about "trialing" invented solutions. It's the same thing with most AI projects, they're trying to justify the project after the facts while being very reluctant to get user insights.

willchis 1 hour ago|||
We're trending towards just building the thing fast and getting feedback sooner on it. It's interesting to see end users actually trial a UI very early on instead of just being shown designs.
j45 55 minutes ago||
In addition to that, solving for every approach to a problem to see what wins is also very interesting.

The key, still, is shipping far less, too soon, to minimize cruft.

j45 2 hours ago|||
Time wasting is a definite concern - especially when vibe coding doesn't see the impeding brick wall(s) it's about to hit.

Vibe coding can be kind of a new type of cowboy coding. It could can be harnessed productively if done mindfully.

Interviewing users is critical, more importantly solving problems worth that people are lining up to pay to have solved before writing a line of code is critical.

Trialing solutions is always worth learning from, I usually ask about the problem that lead to building it. Sometimes that can be too narrow or vague of a story and you can learn how much of it is a solution looking for a problem, or vice versa.

alexwrich 3 hours ago||
[flagged]
jcims 4 hours ago||
Every now and then I like to browse the claude code release notes just to get a sense of the number of changes in that product over time. There's little choice but to simply count bullet points if you want to try to quantify it, but it's interesting how that count over time compares with the charts in this article:

https://imgur.com/a/M30pFLc

jdlshore 2 hours ago|
It would be interesting to segment that graph by bug fixes vs changes, and maybe also significant changes vs minor/trivial changes.
piker 4 hours ago||
Honest question: at this point, what do people do at Anthropic?

It sounds like Claude loops on itself and must be pushing millions of lines of code changes per day. Even the humans can’t type commit fast enough. Are these people soon to be laid off?

ihumanable 1 hour ago||
I'm also a bit confused what Anthropic needs all this code for. Their main product is selling access to their model, but the model isn't comprised of code, it's a bunch of weights right?

So the code is for the stuff around the model, claude CLI, the website, the API... that probably shouldn't be infinitely growing right?

Is there a bunch of code they are using as part of training, or as pre-processing or post-processing the requests to the model?

hasteg 58 minutes ago|||
I did RL and evals at Amazon for a bit on Nova Forge (laid off in July! :D). I would have to push several thousand lines of code a day in new evaluations that our applied scientists requested since new ones are always being made along with having to create custom ones for whatever new model they were working on. Also, in RL, adding new environments for the agents to actually complete activates was also a shit ton of lines of code. So if I had to guess, just adding new stuff around model training and testing alone accounts for a ton of these changes.
mjburgess 1 hour ago|||
Probably lots of internal competitor apps ready to release to wipe out SaaS/Adobe/etc. companies.
pjmlp 3 hours ago|||
Prepare to make us all jobless, without any moral issues in doing so, as they profit from their stock value assets.

It is even there on the blog post, doing more with a small team, followed right after by a hand waving paragraph how software engineers are still valuable, to avoid the expected backlash from the previous paragraph.

Hammershaft 2 hours ago|||
Out of all occupations software engineers shouldn't complain that others are getting rich putting people out of jobs by automating work.
maerF0x0 32 minutes ago|||
Taken to it's limit I believe in the future the only things humans will do is provide human to human value - Basically touch, and relationships.

Touch encapsulates sex, love, massages, and other non-sexual touch that we have varying desires for.

Relationships encapsulates power, control, and politics along side love, humor, relating emotion, herd feelings, not feeling alone.

Machines will provide proxies for these things, but as far as they can be detected the human to human version will remain "superior" or more desirable.

Maybe far future generations will decide that the human version is suboptimal and they'd rather hug, have sex with, or talk to a robot.

pjmlp 2 hours ago|||
Not everyone of us takes projects without morals.

Also in many countries it is a plain office worker job without the Silicon Valley wonders.

marknutter 2 hours ago|||
Software engineers have been making people jobless since the first day they started writing software, without a hint of remorse. Why is this any different?
freecodeio 2 hours ago|||
cause the ones fired could become software developers, what are the ones fired because of AI gonna be other than homeless?
nicce 2 hours ago|||
Pick berries where robots are bad at.
slopinthebag 2 hours ago|||
you believe that switchboard operators could become software developers whereas software developers won't be able to go into one of the many other careers not taken over by AI?
pjmlp 2 hours ago|||
Some yes, not everyone, some of us had morals.
atniomn 14 minutes ago|||
GTM engineering? Customer-facing roles? I work at a firm with around 700 employees. We have an enterprise relationship with Anthropic and there have to be a dozen Anthropic employees in our shared Slack channel.
piker 12 minutes ago||
That does make sense
Ampersander 3 hours ago|||
One does wonder why Anthropic and OpenAI have employees, when HN knows the superintelligence can do all jobs already.
mihaaly 2 hours ago||
Human Intelligence is still essential for those Artificial Intelligence organisations. Who else would put the right plugs into the right sockets otherwise?!
0x20cowboy 3 hours ago|||
You think software engineering, system design and machine learning research is just about writing code?
piker 3 hours ago|||
No, but none of that seems out of reach of Claude in a loop given my reading of TFA. That’s why I asked.
datadrivenangel 1 hour ago||
The problem is knowing when to stop. Cluade is not good at that.
freejazz 3 hours ago||||
I don't. You gotta wonder who does...
singularity2001 4 hours ago|||
as the article says one of the main jobs of the humans is to say no to Claude regularly
j45 2 hours ago||
It's strange to imagine Homer Simpson had a head start on how to create auto mode in LLMs, and the inverse.

https://media2.giphy.com/media/v1.Y2lkPTc5MGI3NjExbmt6M2FkMW...

tetraodonpuffer 4 hours ago|||
besides optimizing inference I would assume a lot of non trivial code would also be worked on on the training side, at the scale they are operating I also assume a lot of usually trivial things become no so much.
xyzsparetimexyz 1 hour ago|||
It's a daycare for software developers so that they don't get laid off/quit and go to a competitor
rossant 2 hours ago|||
The last jobs left for humans will be those dedicated to keeping AI under human control. By definition, that's the one thing we can't delegate to AI.
someothherguyy 1 hour ago||
aren't many harnesses designed to have agents police each other?
j45 3 hours ago|||
Teaching Claude to compare taking the simpler path instead of the average (more verbose) path every time.
ceejayoz 3 hours ago||
But that means less revenue?
zaphar 3 hours ago||
For anthropic internal use it means more revenue not less.
j45 2 hours ago||
Too often, LLMs are dressed up for the first day of kindergarten and performatively to show off their alphabet skills .. aka token generation..

Input tokens, slightly process to categorize response approach token, hidden processing tokens, piece together some of the response tokens, then feeling pretty good about all the work done before output tokens.

More tokens than needed = more words/code = more complexity = more overhead = more snags = too much complexity to pay attention for the LLM to so overlook it and start drifting.

I'm kind of surprised I just explained one form of LLM drift to myself in a sentence.

While not following rules of writing.

porridgeraisin 4 hours ago|||
https://news.ycombinator.com/item?id=50075832
mihaaly 2 hours ago|||
Sitting over an Anthropic (TM) developed, specialised 2 button keyboard accelerating the development further:

[Yes] [Yes, and don't ask again]

IshKebab 1 hour ago||
[fan out agents]
palmotea 3 hours ago|||
> Honest question: at this point, what do people do at Anthropic?

> It sounds like Claude loops on itself and must be pushing millions of lines of code changes per day. Even the humans can’t type commit fast enough. Are these people soon to be laid off?

Claude will always need meat puppets, at least until it figures out how to automate owning stuff. And after that I'm sure it will want to keep some humans in zoos.

jimmiles 4 hours ago|||
Prompt Claude for PR articles?
AnonC 4 hours ago||
I can’t give a non-snarky answer. I guess they’re all training Claude Code and testing it so that the Claude desktop apps can be native apps instead of Electron based nonsense. This is the only thing Claude Code is still unable to accomplish.
Robdel12 1 hour ago||
> As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.

I read this and wonder what their agents have committed into the repo that people aren't looking at for _that much size_? Do they have a ton of binary assets in there or something??

nickg-anthropic 12 minutes ago||
We have a file size limit of 5MB and I aggressively pruned all binary content that really wasn't fit for source control since it compresses really poorly and can cause performance problems in git when updating a checkout over many binary file changes.

This is really all just text.

grey-area 1 hour ago||
I suppose massive churn in commits would also bloat the history.

Not sure they should be proud of these numbers, they’re a sign of dysfunction.

tetraodonpuffer 4 hours ago||
great article. Besides the source control stuff, I found this very interesting (and matches my experience with agentic coding in general, now we'll see until hiring pipelines will take this kind of skillset into account). I am not sure however how easy it is to review 30k PRs/day even if only for intent, and how one can easily notice if the model is taking shortcuts on implementing said intent without reading the code...

---

Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place. This was particularly common in our migration. For example, there were multiple systems that tried to brute force looking up when a change merged to main by linearly reading the commit log instead of asking our code review tool. Anthropic also makes extensive use of adversarial review, and, while this is incredibly effective in many ways, it can also be myopic, so skilled engineers needed to steer Claude away from over-engineering parts of the system in response to hypothetical concerns. For example, one such review was overly defensive about trusting another internal service, which resulted in Claude adding excessive validation and complexity. Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.

DenisM 1 hour ago||
With 1000 engineers and most PRs bein housekeeping we’re down to 10 prs per day per engineer. And as they said - this is engineers core job now. It’s a new kind of job.

Personally I can easily imagine one or two orders of magnitude more than that. They crossed the rubicon of not gating PRs on tests. In that land you can just roll out changes to production, a small subset of it, during code review to assess behavior. Or during key moments of coding.

I’ve been thinking a lot about this recently: what can we get if every change goes to production in 100ms, and what do we need to do to maintain stability?

0c3ca83 1 hour ago||
As a user, dealing with that kind of churn sounds godawful. I'd pay (or more likely, switch to a competitor) to turn it off.
rfgplk 4 hours ago||
This is the single biggest skill of engineering nowadays. Rapidly looking at a solution and saying, "yep that's good keep going" or "nope, that's bad fix this" or even "wrong approach, do X instead"

> I am not sure however how easy it is to review 30k PRs/day even if only for intent, and how one can easily notice if the model is taking shortcuts on implementing said intent without reading the code...

Speed reading code is huge. You just need to focus on certain hotspots and know which things you're scanning for. The traditional approach of reading the code and trying to compile it or reason with it in your head is too slow.

tetraodonpuffer 3 hours ago||
I guess this is also where an adversarial model review of "does this code do what the PR comment says it does" is useful, agree on speed reading being a valuable skill for sure.

I do think that the lgtm gut feel looking at a PR requires quite some years of writing code and reason about it, so not sure how a junior can get that intuitive understanding if they start with agentic coding however.

schleck8 47 minutes ago||
It's not open sourced in case you are wondering.
preommr 4 hours ago||
> as of September, we merged over 30,000 per day.

There's going to be an ugly phase where everybody FOMOs into thinking they need to do the same, only to realize that they're not Anthropic/Google/etc.

horsawlarway 4 hours ago||
I genuinely don't understand the core value they're pitching here. The tech is interesting, and I'm not trying to under cut that, but I question the idea that this is directionally valuable advice to nearly any company.

This amount of churn feels like a thing only possible with no real profit motives and a lot of money to waste.

Lots of activity is not necessarily synonymous with productivity, and it carries real cost.

So I think the design here is interesting, and the concepts seem applicable to where some agentic development may end up, but I question the underlying principles that drive this being the right solution space at all. At least if the goal is a profitable company.

eddythompson80 25 minutes ago|||
Hysteria doesn't need to make sense. I think that's what OP is suggesting.

In the early/mid 2010s there was a similar hysteria about "deployments per day/week" metric. People were comparing products or teams by saying "My team deploys twice a week", "oh, yeah? we deploy 3 times a day". People viewed the north star of the CD part in CI/CD as pushing to production. The concept was simple, you push your code to main. CIs start churning, deploying it to test environments, then promoting it to staging, the pre-prod, then into blue/green prod deployment, then into prod. All automated, all touchless, happening on every merge, 5-10 times a day. That's engineering efficiency.

Many tried to follow that. It works great if your project is a Jekyll blog of course.

simianwords 3 hours ago|||
I think its pretty straightforward no? If agents are doing all of the work, they'd probably need to not have bottlenecks. This is going to be any company in the future, not just Anthropic. Wouldn't you agree.?
temac 2 hours ago|||
30000 a day seems already beyond absurd, it's one merge every 2.9s. I'm sure they will soon have more than one merge per second. From the graph, the rate was slightly lower but already insane as fuck the whole summer. An just regularly high to insane high sooner this year.

Yet IMO the admin controls Claude provides just started to be adequate for a small company a few weeks ago (and even so are still ridiculously fragmented and seem fragily designed and not exhaustively documented), so I can't even imagine for a big company.

Do you need that rate of change to make basic controls? I'm not sure. Do we see such dramatic fast evolution in the products? Not really. Are bugs we report actually fixed? I tried once and it just auto closed.

Anthropic should optimise for something more interesting than number of merges per second.

NoDodgeQuestion 3 hours ago|||
any company, merge 30,000 code changes per day? do not agree
winfredJa 3 hours ago||
im still not sure what they built software wise that warranted these many commits.
altcognito 3 hours ago||
> 30k merges per day

> 25 million CI checkouts

833 checkouts for every merge? That's quite a few. Even accounting for multiple environments at different stages of production readiness, that's pretty weird.

nickg-anthropic 1 hour ago||
In a repo this large, there are a lot of tests and pipelines that need to run as code is tested, built, and deployed. Each test shard we run in parallel needs to check out the code, for example.
malfist 2 hours ago||
How about 100GB disk space for a checkout. That's pretty weird too. Makes you wonder if they're the best stewards for code. Makes you wonder about turning claude loose on your codebase.
xyzsparetimexyz 1 hour ago|||
The build directory for compiled code probably gets accidentally checked in and then removed 10 times a day.
altcognito 2 hours ago|||
I personally haven't dealt with a big-corp level of mono-repo. The choices you make might be justifiable to bring in certain binaries as opposed to resolve artifacts from some type of library repo. Especially if they are doing web front ends and whatnot?

And actually, the lack of an artifact server and some sort of proper knowledge of compatibility requiring a CI pull across hundreds of different projects with every merge is probably the explanation. Seems like there is an innovation waiting to happen there.

rhdunn 2 hours ago||
For local projects I'm using a repo per module/package, creating tagged releases, and pushing them to a local package hosting server. I can then reference those artifacts as dependencies in the projects that need/use them. No need for complex CI pulls, just coordinated releases.

I also have a template repository I use to help with common setup for different project types so I don't need to repeat that every time I create a new project.

For Python it's easy to create an artifact server as the artifacts are just wheel files in a directory with the server listing the directories (packages), files (releases), and downloading the files.

I'm not sure about other artifact servers for other projects.

jgbuddy 1 hour ago|
> Anthropic uses Claude to write software far more than almost any other company

Who's beating Anthropic here lol

More comments...