Top
Best
New

Posted by chwtutha 1 day ago

Vouch(github.com)
https://x.com/mitchellh/status/2020252149117313349

https://nitter.net/mitchellh/status/2020252149117313349

https://github.com/ghostty-org/ghostty/pull/10559

855 points | 385 commentspage 9
baq 16 hours ago|
Central karma database next, please. Vouch = upvote, denounce = downvote
skeeter2020 17 hours ago||
Doesn't this just shift the same hard problem from code to people? It may seem easier to assess the "quality" of a person, but I think there are all sorts of complex social dynamics at play, plus far more change over time. Leave it to us nerds to try and solve a human problem with a technical solution...
mjr00 17 hours ago|
> Leave it to us nerds to try and solve a human problem with a technical solution...

Honestly, my view is that this is a technical solution for a cultural problem. Particularly in the last ~10 years, open source has really been pushed into a "corporate dress rehearsal" culture. All communication is expected to be highly professional. Talk to everyone who opens an issue or PR with the respect you would a coworker. Say nothing that might offend anyone anywhere, keep it PG-13. Even Linus had to pull back on his famously virtiolic responses to shitty code in PRs.

Being open and inclusive is great, but bad actors have really exploited this. The proper response to an obviously AI-generated slop PR should be "fuck off", closing the PR, and banning them from the repo. But maintainers are uncomfortable with doing this directly since it violates the corporate dress rehearsal kayfabe, so vouch is a roundabout way of accomplishing this.

zbentley 16 hours ago|||
What on earth makes you think that denouncing a bot PR with stronger language would deter it? The bot does not and cannot care.

If that worked, then there would be an epidemic of phone scammers or email phishers having epiphanies and changing careers when their victims reply with (well deserved) angry screeds.

mjr00 15 hours ago||
I didn't mean the "fuck off" part to be quite verbatim... this ghostty PR[0] is a good example of how this stuff should be handled. Notably: there's no attempt to review or provide feedback--it's instantly recognized as a slop PR--and it's an instant ban from repo.

This is the level of response these PRs deserve. What people shouldn't be doing is treating these as good-faith requests and trying to provide feedback or asking them to refactor, like they're mentoring a junior dev. It'll just fall on deaf ears.

[0] https://github.com/ghostty-org/ghostty/pull/10588

zozbot234 15 hours ago||
Sure, but that pull request is blatantly unreviewable because of how it bundles dozens of entirely unrelated commits together. Just say that and move on: it only takes a one-line comment and it informs potential contributors about what to avoid if any of them is lurking the repo.
jack_pp 15 hours ago|||
One problem with giving any feedback is that it can automatically be used by an agent to make another PR.
zozbot234 14 hours ago||
If they immediately make another low-quality PR that's when you ban them because they're clearly behaving like a bad actor. But providing even trivial, boilerplate feedback like that is an easy way of drawing a bright line for contributors: you're not going to review contributions that are blatantly low-quality, and that's why they must refrain from trying to post raw AI slop.
mjr00 14 hours ago|||
Sounds like we're largely saying the same thing. Open source maintainers should feel empowered to say "nope, this is slop, not reading, bye" and ban you from the repo, without worrying if that seems unprofessional.
zozbot234 14 hours ago||
If you explicitly say "this is unreviewable junk, kthxbye" there's nothing unprofessional about it. But just blaming "AI slop" runs into the obvious issue that most people may be quite unaware that AI will generate unreviewable junk by default, unless it's being very carefully directed by an expert user.
verdverm 16 hours ago||||
> Particularly in the last ~10 years ...

This is maturation, open source being professional is a good sign for the future

zozbot234 17 hours ago|||
I disagree. The problem with AI slop is not so much that it's from AI, but that it's pretty much always completely unreadable and unmaintainable code. So just tell the contributor that their work is not up to standard, and if they persist they will get banned from contributing further. It's their job to refactor the contribution so that it's as easy as possible to review, and if AI is not up to the task this will obviously require human effort.
mjr00 16 hours ago|||
You're giving way too much credit to the people spamming these slop PRs. These are not good faith contributions by people trying to help. They are people trying to get pull requests merged for selfish reasons, whether that's a free shirt or something to put on their resume. Even on the first page of closed ghostty PRs I was able to find some prime slop[0]. It is a huge waste of time for a maintainer to nicely tell people like this they need to refactor. They're not going to listen.

edit; and just to be totally clear this isn't an anti-AI statement. You can still make valid, even good PRs with AI. Mitchell just posted about using AI himself recently[1]. This is about AI making it easy for people to spam low-quality slop in what is essentially a DoS attack on maintainers' attention.

[0] https://github.com/ghostty-org/ghostty/pull/10588

[1] https://mitchellh.com/writing/my-ai-adoption-journey

zozbot234 16 hours ago||
If you can immediately tell "this is just AI slop" that's all the review and "attention" you need; you can close the PR and append a boilerplate message that tells the contributor what to do if they want to turn this into a productive contribution. Whether they're "good faith contributors trying to help" or not is immaterial if this is their first interaction. If they don't get the point and spam the repo again then sure, treat them as bad actors.
michaelt 16 hours ago||
The thing is, the person will use their AI to respond to your boilerplate.

That means you, like John Henry, are competing against a machine at the thing that machine was designed to do.

bpavuk 16 hours ago|||
...and waste valuable time reviewing AI slop? it looks surprisingly plausible, but never integrates with the bigger picture.
danilocesar 15 hours ago||
Wait until he finds out about GPG signing parties in the early 2000s.
a-dub 14 hours ago||
this highlights the saddest thing about this whole generative ai thing. beforehand, there was opportunity to learn, deliver and prove oneself outside of classical social organization. now that's all going to go away and everyone is going to fall back on credentials and social standing. what an incredible shame for social mobility and those who for one reason or another don't fit in with traditional structures.
boltzmann-brain 14 hours ago||
Vouch is a good quick fix, but it has some properties that can lead to collapsed states, discussed in the article linked here: https://news.ycombinator.com/item?id=46938811
a-dub 14 hours ago||
it's also going to kill the open web. nobody is going to want to share their ideas or code publicly anymore. with the natural barriers gone, the incentives to share will go to zero. everything will happen behind closed doors.
tolerance 13 hours ago||
You could argue that this could increase output to the open web: outsiders still need a place to clout chase.
boltzmann-brain 13 hours ago||
GitHub has never been a good method of clout chasing. in decades of being in this industry, I've seen < 1% of potential employers care about FLOSS contributions, as long as you have some stuff on your GH.
senko 13 hours ago|||
The origin of the problems with low-quality drive-by requests is github's social nature[0]. AI doesn't help, but it's not the cause.

I've seen my share of zero-effort drive-by "contributions" so people can pad their GH profile, long before AI, on tiny obscure projects I have published there: larger and more prominent projects have always been spammed.

If anything, the AI-enabled flood will force the reckoning that was long time coming.

[0] https://news.ycombinator.com/item?id=46731646

patcon 13 hours ago|||
I feel this is a bit too pessimistic. For example, people can make tutorials that auto-certify in vouch. Or others can write agent skills that share etiquette, which agents must demonstrate usage of before PRs can be created.

Yes, there's room for deception, but this is mostly about superhuman skills and newcomer ignorance and a new eternal September that we'll surely figure out

potsandpans 14 hours ago|||
> that's all going to go away and everyone is going to fall back on credentials and social standing.

Only if you allow people like this to normalize it.

yencabulator 14 hours ago|||
.. all revolving around a proprietary Microsoft service.

Support Microsoft or be socially shunned?

mitchellh 13 hours ago||
Vouch is forge-agnostic. See the 2nd paragraph in the README:

> The implementation is generic and can be used by any project on any code forge, but we provide GitHub integration out of the box via GitHub actions and the CLI.

And then see the trust format which allows for a platform tag. There isn't even a default-GitHub approach, just the GitHub actions default to GitHub via `--default-platform` flag (which makes sense cause they're being invoked ON GITHUB).

yencabulator 13 hours ago|||
Define "platform".

So I can choose from github, gitlab or maybe codeberg? What about self-hosters, with project-specific forges? What about the fact that I have an account on multiple forges, that are all me?

This seems to be overly biased toward centralized services, which means it's just serving to further re-enforce Microsoft's dominance.

mitchellh 13 hours ago||
It's a text string, platform can be anything you want, then use the vouch CLI (or parse it yourself) to do whatever you want. We don't do identity mapping, because cross-forge projects are rare and maintaining that would centralize the system and its not what we're trying to do. The whole thing is explicitly decentralized with tiny, community specific networks that you build up.
potsandpans 12 hours ago|||
I would rather stop contributing to open source rather than interact with your gatekeeping social experiment.
mitchellh 11 hours ago||
That’s fine and doesn’t bother me one bit.
potsandpans 11 hours ago||
Tracks. You don't care about the open source community.
mitchellh 11 hours ago||
No, that’s quite a jump. I just respect whatever your preferences are.
cyanydeez 13 hours ago|||
argueably, the years 2015-2020, we should have gone back to social standing.
bicx 14 hours ago|||
I guess you could say the same about a lot of craft- or skill-based professions that ultimately got heavily automated.
siva7 13 hours ago||
It also marks the end of the open source movement as the value of source code has lost any meaning with vibe coding and ai.
rvz 17 hours ago||
This makes sense for large-scale and widely used projects such as Ghostty.

It also addresses the issue in tolerating unchecked or seemingly plausible slop PRs from outside contributors from ever getting merged in easily. By default, they are all untrusted.

Now this social issue has been made worse by vibe-coded PRs; and untrusted outside contributors should instead earn their access to be 'vouched' by the core maintainers rather than them allowing a wild west of slop PRs.

A great deal.

emeraudelinton 7 hours ago||
[dead]
enterprisetalk 1 day ago||
[dead]
fcantournet 11 hours ago||
This looks like a fairly typical engineer's solution to a complex social problem: it doesn't really solve the problem, introduces other issues / is gameable, yet unlikely to create problems for the creator. Of course creator answers any criticism of the solution with "Well make something better". That's not the point: this is most likely net negative, at least that is the (imo well supported) opinion of critics. If the cons outway the pros, then doing nothing is better than this.
dang 6 hours ago||
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

https://news.ycombinator.com/newsguidelines.html

wayeq 10 hours ago|||
did you have any actual criticism?
Bayko 9 hours ago||
cons to YOU outway the pros. pros to HIM outway the cons.
Verlyn139 9 hours ago||
[dead]
returnInfinity 1 day ago|
[flagged]
More comments...