Posted by phmx 5 days ago
"With GC.stress = true, the GC runs after every possible allocation. That causes immediate segfaults because objects get freed before Ruby can even allocate new objects in their memory slots."
That would seem to indicate a situation so broken that you can't expect anything to work reliably. The wrong-value situation would seem to be a subset of a bigger problem. It's like finding C code that depends on use-after-free working and which fails when you turn on buffer scrubbing at free.
Million-to-one bugs are not only real but high enough to matter, depending on which million. Many years ago I had a rare bug that corrupted timestamps in the logs, with an emperical probability of about one to 3--5 million (IIRC). Turned out that that seemingly benign bug was connected to a critical data corruption issue with real consumer complaints. (I have described this bug in detail in the past, see my past comment for details.)
OTOH I'm a bit surprised he didn't pull back earlier and suggest to his user to update to the latest version though and let him know.
Never wrote it up externally because it was already solved and "Debian updates to existing releases are so rare that you really want to pay attention to all of them" (1) was already obvious (2) was only relevant to a really small set of people (3) this somewhat tortured example wasn't going to reach that small set anyway. (Made a reasonable interview story, though.)
If you cannot be bothered to write something, why on God's good earth would you expect anyone to be bothered to read it?
> No warning. No error. Just different methods that make no sense.
> This is why write barriers exist. They're not optional extras for C extension authors. They're how you tell the garbage collector: "I'm holding a reference. Don't free this
It's all ChatGPT LinkedIn and Instagram spam type slop. An unfortunate end to an otherwise interesting writeup.
They like to code. They don’t like to write.
I’m not excusing it, but after you asked the question the conclusion seems logical.
People like reading LLM slop less than either of those. So it should become a common understanding not to waste your (or our) time to "write" this. It's frustrating to give it a chance then get rug-pulled with nonsense and there's really no reason to excuse it.
I would spend similar time debugging this if I were the author. It's a pretty serious bug, a non obvious issue, and would be impossible to connect to the ffi fix unless you already knew the problem.
I’m not saying there’s no AI here. I am asking for some evidence to back up the claim though.
The reason I don't usually bother to bring these specific things up is that I already know the response, which is just going to be you arguing that a human could have written this way, too. Which is true. The point is that if you read the collective whole of the article, it is very clear that it was composed with the aid of AI, regardless of whether any single part of it could be defensibly written by a human. I'd add that sometimes, the writing of people who interact heavily with LLMs all day starts to resemble LLM writing (a phenomenon I don't think people talk enough about), but usually not to this extent.
This doesn't mean that the entire article was written by an LLM, nor does it mean that there's not useful information in it. Regardless, given the amount of low effort LLM-generated spam that makes it onto HN, I think it is fairly defensible to use "this was written with the help of an LLM, and the person posting it did not even bother to edit the article to make that less obvious" as a heuristic to not bother wasting more time on an article.
“not A, not B, not C” and “not A, not B, but C” are extremely common constructions in general. So common in fact that you did it in this exact reply.
“This doesn't mean that the entire article was written by an LLM, nor does it mean that there's not useful information in it. Regardless, given the amount of low effort LLM-generated spam that makes it onto HN, I think it is fairly defensible”
> The style is list-heavy, including lists used for conditionals, and full of random bolding, both characteristic of AI-generated text
This is just blogspam-style writing. Short snippets that are easy to digest with lists to break it up and bold keywords to grab attention. This style was around for years before ChatGPT showed up. LLMs probably do this so much specifically because they were trained on so much blog content. Hell I’ve given feedback to multiple humans to cut out the distracting bold stuff in their communications because it becomes a distraction.
My reply wasn't an instance of this syntactic pattern, and the fact that you think it's the same thing shows that you are probably not capable of recognizing the particular way in which LLMs write.
The thing is, your premise is that you can identify certain patterns as being indicative of AI. However, those exact same patterns are commonly used by humans. So what you’re actually claiming is some additional insight that you can’t share. Because your premise does not hold up on its own. What you were actually claiming is “I know it when I see it”.
Let me give you a related example. If you go to any of the “am I the asshole” subreddits, you will encounter the exact same story format over and over: “Other person engages in obviously unacceptable behavior. I do something reasonable to stop the unacceptable behavior. People who should support me support other person instead. Am I the asshole?” The comments will be filled with people either enraged on behalf of the author or who call it AI.
The problem with claiming that it’s AI is that the sub was full of the exact same garbage before AI showed up. The stores have always been the same bullshit rage bait. So it’s not technically wrong to say it looks like AI, because it certainly could be. But it could also be human generated rage bait because it’s indistinguishable. My guess is that some of the sub is totally AI. And a chunk of it is from human humans engaged in shitty creative writing.
When you look at generic click-bait/blogspam patterns that humans have been using for decades now and call it AI, all you’re doing is calling annoying blog writing AI. Which it could be, but it could also not be. Humans absolutely write blogs like this and have for longer than LLMs have been widely available.
> My reply wasn't an instance of this syntactic pattern, and the fact that you think it's the same thing shows that you are probably not capable of recognizing the particular way in which LLMs write.
It was absolutely an example of the pattern, just more wordy. Spare me the ad hominem.
Your “you couldn’t understand” and “obvious to us” stuff is leaning into conspiracy theory type territory. When you believe you have some special knowledge, but you don’t know how to share it with others, you should question whether that knowledge is actually real.
LLMs simply don't generate the syntactic pattern I used consistently, but they do generate the pattern in the article. I'm not really sure what else to tell you.
The rest of your post isn't really that interesting to me. You asked why nobody was giving specific examples of why it was generated. I told you some of the specific reasons we believe this article was generated with the assistance of an LLM, mentioned that the reason people don't typically bother to bring it up is that we know people who demand this sort of thing typically claim without evidence that humans write in the exact same way all the time, and you proceeded to do exactly that. Next time you don't get a response when you ask for evidence, consider that it might be because we don't particularly want to waste time responding to someone who isn't interested in the answer.
I feel like the “this is AI” crowd is getting ridiculous. Too perfect? Clearly AI. Too sloppy? That’s clearly AI too.
Rarely is there anything concrete that the person claiming AI can point to. It’s just “I can tell”. Same confident assurance that all the teachers trusting “AI detectors” have.
I have opened a wager r.e. detecting LLM/AI use in blogs: https://dkdc.dev/posts/llm-ai-blog-challenge/
These ai hunters are like the transvestigators who are certain they can always tell who’s trans.
Also, you do realize that writing is taught in an incredibly formulaic way? I can't speak to English as second language authors, but I imagine it doesn't make it easier.
> I won't actually make this bet!
> But if I did make this bet, I would win!
???
see other comment though, the point is that assessing quality of content on whether AI was used is stupid (and getting really annoying)
just one scenario, I write 100 rather short, very similar blog posts. run 50 through Claude Code with instructions “copy this file”. have fun distinguishing! of course that’s an extreme way to go about it, but I could use the AI more and end up at the same result trivially
obviously if a million dollars are on the line I’m going to do what I can to win. I’m just pointing out how that can be taken to the extreme, but again I can use the tools more in the spirit of the challenge and (very easily) end up with the same results
That's a laughable response.
this wager is a thought exercise to demonstrate that. want to wager $1,000,000 or think you’ll lose? if you’ll lose, why is it ok to go around writing “YoU uSeD aI” instead of actually assessing the quality of a post?
If you could actually identify AI deterministically you would have a very profitable product.
I find it interesting that you believe this claim is wildly conspirational, or that you think the difficulty of reliably detecting AI generated text at scale is evidence that humans can't do pretty well at this much more limited task. Do you also find claims that AIs are frequently sycophantic in ways that humans are not, or that they will use phrases like "you're absolutely right!" far more than a human would unless prompted otherwise (which are the exact same type of narrow claim) similarly conspirational? i.e., is your assertion that people would have difficulty differentiating between a real human's response to a prompt and Claude's response to a prompt when there was no specific pre-prompt trying to control the writing style of the response?
Is this the new "looks shopped. I can tell by the pixels."?
From what I've seen doesn't it take a particularly strong reason for the entire article to get dismissed
Pro-tip: re-read your comment before you submit and take out the bits that make you sound like an asshole.