Detecting LLM-Generated Texts with “Classical” Machine Learning

Posted by uneven9434 5 hours ago

Detecting LLM-Generated Texts with “Classical” Machine Learning(blog.lyc8503.net)

106 points | 78 commentspage 2

teeray 4 hours ago|

The problems are simply too great if an LLM detector has any false positives at all. Imagine how soul-crushing writing an entire dissertation by hand and having it rejected because some “good enough” LLM detector decides you write too much like an AI.

rayval 3 hours ago||

As I recall, a few years ago (in the era of first generation LLMs), a professor in Texas used an anti-plagiarism tool that flagged more than one-third of the class using AI in an exam, and used that finding to give them a failing grade.

If memory serves, one student objected strenously and ran the professor's own work (published 10 years earlier) into the same tool and it flagged that work as AI-generated.

EDIT: HN item from June 2023 https://news.ycombinator.com/item?id=36215823

pixl97 2 hours ago||

Exactly. The more corporate and proper you tend to speak, the more likely it's to classify you as an LLM. It's like the classifiers want us to talk like trash at their current rate. This seems to be really problematic for ESL speakers/typers that may have been trained on a smaller, more proper subset of the language.

dmurvihill 3 hours ago||

It depends on the application. Dissertation? Hell naw. Blog post? Absolutely, run it through that thing.

teeray 2 hours ago||

The problem is that ed-tech is absolutely ravenous for an LLM detector and would rather use snake oil than accept that it might not be possible.

richard_chase 3 hours ago||

Am I the only who largely enjoys the output of LLMs more than most stuff written by humans? I find myself coming back to old chats with ChatGPT frequently because the output is amazing.

therealdrag0 3 hours ago|

I wouldn’t go that far… but it can be kinda like Wikipedia, clean and readable.

XiphiasX 4 hours ago||

Anything too “clever” and “snappy” = instaLLM

hasteg 4 hours ago|

This is also how I pretty much filter LLM generated text in my head.

cyanydeez 5 hours ago||

today, sure.

Tomorrow, the LLMs will be training the humans thought patterns that will directly start skewing their natural writing.

Generation alpha is going to have a lot of trouble if we keep perpetuating the myth that you can really interpret text in an ongoing fashion.

pixl97 2 hours ago|

I think you're about a year late for this revolation.

https://www.washingtonpost.com/opinions/2025/08/20/chatgpt-c...

cyanydeez 2 hours ago||

I'm not late if people constantly put effort into finding LLM text, or every other comment on hacker news is either about something being LLM generator.

pixl97 2 hours ago||

After seeing comments on hacker news attempt to call an article from 2015 as generated by an LLM, I have very little faith in commenters having any ability in actually detecting AI written text.

And that's just one particularly egregious case I remember. Posters that are technical writers or use English properly get called bots quite commonly when their post history shows a writing style going back over a decade.

But now that LLMs are causing a language drift in English users our filters of "that's an LLM" will become even more useless.

metalman 3 hours ago|

there is not much point in detecting LLM generated text, in that humans are useing info from LLM's, but obfusicting it's origin, with there own garble, along with purely human garble, and almost(but not quite) human LLM product meaning that the threshold for rejecting "data" must be lowered, which personaly means a very very low tollerance for wierdness, except where it can yield imediate possitive cash flow for the rest I do my own research and verification thank you very much