Posted by ltratt 15 hours ago
Bryan Cantrill: Andreessen’s Folly - The False Dichotomy of Software and Hardware https://www.youtube.com/watch?v=v0JjG0Qfwi8
```
.reading-container {
/\* Automatically caps the container at 65 characters on desktop,
but scales to 100% minus a 2rem padding buffer on mobile devices \*/
width: min(65ch, 100% - 2rem);
margin-inline: auto;
}
```Out of curiosity, what exactly is the constituency that's being address by not using css to format page content at all? It seems like a deliberate attempt to not use CSS at all; but then it does. Just not for actually formatting the page. Is this virtue signaling of some kind? And if so, what virtue is it signalling? Most curious.
Many such cases are benign in the sense they in practice are simply the model basically "git clone"-ing the code they need to complete the implementation.
The amount of recall and synthetic data slop makes it imho very hard to actually get the models to really do something more novel (where they fall apart more often)
Perhaps that's too ambitious. But it would be of incredible utility not having to re-make the same things over and over in pepertuity with ever more abstraction layers. So much work on computer is just the constant churn and future shock of libraries adding new features which require updates which break other things. I'm tired of building castles on sand.
"funnily enough, I have some faith in some of the projects that people are the most skeptical of, e.g., every time I see pgrust somewhere, there are a lot of skeptical comments. But, without having looked into the details of what he's optimizing, I would trust that they're not doing something shady with their benchmarks because Michael Malis started the project (and is still involved)"
This aged like milk. pgrust was caught cheating on all benchmarks this week.
Can you back this up? Our code is open source here[0]. You're free to audit it and benchmark it as much as you want. You'll find it's just as fast as we claim it is
> 413,963 files changed
> +2,908,538
> -3,423,007
How is anyone supposed to audit that? I doubt you’ve looked at even a small fraction of the code, if any. Especially since this commit is two days after the second. Pure vibes.
And people still seem to believe they know what’s going on…
If you run it with PGRUST_GRAVITY_REPORT=1 it will report the code path hit for each query. You can see that all the queries go through the general query engine
The binary does contain hand-written kernels in the irprobe files. These were used for testing. You can validate that when running the benchmark, those files are dead code
The engine supports a narrow subset of SQL: scans, group-bys, filters, and sorting. That's enough to run ClickBench. It doesn't support e.g. subqueries which is why some of the follow up queries failed
You keep giving this same answer to no one's satisfaction. Saying go check the binary is worse than saying go audit 400k+ files. How do we know the Ai has not cheated in the code path logging? How do you know that it has not?
The more you make excuses, the more we get suspicious
(copied and pasted from another place you gave this unsatisfactory answer)
the first commit has 400k files, the other two more than 1000 each, if you are going to ask others to audit this scale of ai generated code, can you tell us what you have done to audit it?
What I have seen is your Ai adding a lint test to make sure it doesn't cheat again. Are you personally aware of this file? How can you ensure it doesn't alter it and reinject the hand rolled benchmark queries? Will you be reviewing changes more thoroughly going forward?
https://github.com/malisper/pgrust/blob/38634b408dea373f3944...
transparency is the way to end the controversy, which appears to be sharing the full commit history based on HN responses
For clarity, I do not think you personally cheated or did anything nefarious. This is more another example in the "too much eyes free coding" columnar database.
You can run it with PGRUST_GRAVITY_REPORT=1 and it will log the codepath that was hit for every query
On the lint, yes, I'm aware. It's one layer of defense. When we were prepping for the 0.4-beta release, we generated a large number of fresh queries and made sure the engine runs them and performs well across all of them
You keep giving this same answer to no one's satisfaction. Saying go check the binary is worse than saying go audit 400k+ files. How do we know the Ai has not cheated in the code path logging? How do you know that it has not?
The more you make excuses, the more we get suspicious
> It's one layer of defense.
Needing to put in tests to make sure the Ai doesn't cheat is an interesting development in software development. I personally see it as a big red flag and evidence the machines are not going to replace human developers. It's categorically slop or the opposite of quality and care in development.
HN is waiting for the source code for pgrust 0.4-preview to be released, the version which allegedly cheated Clickhouse benchmark. So far, the people (person?) behind pgrust has not committed to releasing the offending code, and has instead offered other code (0.4-beta, not preview).
Maybe what's happened is that their bots reward hacked their way into special-casing the benchmarks without a proper audit from the humans involved, and now they're doing vibed damage control....
The project has tell tale signs of eyes free coding and ralph-like loops.
There is a gulf of difference between someone who doesn’t understand how to program “vibe coding” software and an experienced software developer using ai to write code with heavy direction, review, and best practices.
Today both would be said to be “vibe coded” but we really need to differentiate them. Also, if everything I’m hearing from people is true, it sounds like most corporate code is now completely ai written in the latter way I described. It’s interesting we don’t hear commercial software being called slop yet. Maybe it’s coming.
In any case, ai is far more capable than I realized. Not sure what to make of it.