I tried the detector on several of my own Stack Overflow answers, and the fourth or fifth one I tried was flagged as "85 % machine / likely machine-written" with the default settings, well above the default sensitivity threshold of 42%. When I turn off the option to strip LaTeX commands (the source is Markdown), that jumps to 96% machine.

Admittedly, a Stack Overflow answer is somewhat outside the realm of scientific writing, so it's still possible that the detector may be accurate within that domain. That said, it's a cautionary tale on the hazards of applying classifiers like this outside of the domain that they were trained on.

I work with several researchers (primarily working on consensus protocols etc).

We were discussing research in general and I asked them: "Do you prefer the writing of the papers or the research?"

They, almost unanimously, agreed that they preferred the research. This makes sense as if they preferred writing they probably would have chosen another profession.

I say this b/c having LLMs available to turn research diagrams, code etc into a paper (or at least the starting point of a paper) will probably lead to MORE quality research papers. This is b/c I'm sure there was some friction in a researcher's mind of "I would love to do the research on this but don't want the trouble of writing the paper".

Put another way: on a 2D plot with one axis being the skills as a researcher and the other being hatred of writing, LLMs may "unlock" the people high on both axes to get more papers out.

Post Script: I agree that this could also lead to more BAD papers but the net may turn out to be positive in the long run.

Something I've thought about a lot is that there is having someone with some domain knowledge or reason to care a lot about a particular issue spend a bunch of tokens and cycles on it until something useful comes out the other end. The most obvious ones are the math problems that have been coming out and help push the frontier of various areas of math. Another example is taking all of the public NYC open data ecosystem and crunching it to get some value which I have spent a lot of time and tokens on but not found a great medium to share.

The question is just how to organize these outputs and conclusions in a way that is consistently reproducible and also how to correct errors or remove LLM nonsense where it refuses to take a position on something.

Before it made sense to do this in papers but it feels like we need something like a paper format.. that is fully reproducible ideally and optimized for aggregating knowledge in a better way. I.e. before a person spent months on one of these and there was just more filtering, and the output itself was a clear signal of time spent and effort that no longer exists.

I get 0% (accurately) on my latest paper. Not super surprised, as I intentionally avoid some LLM-isms that I used to use because I don't want reviewers to have even the slightest indication that text is LLM-generated (even if in principle I'm not opposed to polishing or even wholesale generating academic text if it can convey the original research well, especially for non-native speakers).

I don't think the problem is as bad as a naive reading of this article suggests. I'm highly skeptical that anywhere near 65% of recent CS papers that I've read (mostly systems papers) are substantially AI-written. I threw some recent papers I've read into the system and they come back as 0-7%.

To corroborate this - I found pretty similar rates of AI-flagged papers over time from running pangram on ArXiv (abstracts) in a physics subfield, which is currently at about 25% as of April (write up here: https://peterse.github.io/2026/06/15/The-rising-tide-pt1.htm...).

Its not clear from your writeup what threshold needs to be reached to be classified as "machine written". A preprint where half the text is human and half is 100% AI should be a different category than a preprint where 100% of the text is AI-assisted.

Also its cool that you're making the detector available. When you say "cheap to run", do you know how this compares to pricing for a commercial detector pangram or GPTZero?

Is it just me, or does this article itself read as AI generated?.. Ironically, I tried running the text of this article through their own classifier, and pretty much all of it was highlighted in red. I'm not sure what to make of this. Is this some kind of intentional irony?

I run most professional statements and articles through LLMs before submission. This helps correct grammar and improves accessibility through better syntax (because I write exactly how I think).

The article doesn't seem to mention consideration of AI for polishing human work.

I just generated some docs for a lib I'm writing and it says: > 9 % machine likely human-written

I’m very skeptical of these results. I run a pretty large group paper website on top of arXiv and we run pangram on papers. The numbers are not nearly this high.

One thing I see a lot is papers flagged as AI because they include llm rollouts in the paper as examples.

I ran two papers and three blog posts of mine through here, and all but one (correctly) flagged as 0-1% machine written. The other (human written) blog post was 20%. Pretty good afaict!