Top
Best
New

Posted by dw64 4 days ago

Updated practice for review articles and position papers in ArXiv CS category(blog.arxiv.org)
497 points | 237 commentspage 3
jruohonen 3 days ago|
One thing I forgot to speculate: a position paper on DEI and Cornell University...
zekrioca 4 days ago||
Two perspectives: Either (I) LLMs made survey papers irrelevant, or (II) LLMs killed a useful set of arXiv papers.
iberator 4 days ago||
Simple solution: criminalize posting AI generated publications IF NOT DISCLOSED CLEARLY.

Lets say 50000€ fine, or 1 year in prison. :)

tasuki 4 days ago||
Would you like to have to prove your comment wasn't written by an AI or would you rather go to prison?
deltaburnt 4 days ago||
Literally everything will say AI generated to avoid potential liability. You'll have a "known to the state of California to cause cancer" situation.
j45 4 days ago||
Have the papers gotten that good or bad?
candiddevmike 4 days ago||
I've seen quite a few preprints posted on HN with clearly fantastical claims that only seem to reinforce or ride the coattails of the current hype cycle. It's no longer research, it's becoming "top of funnel thought leadership".
nunez 4 days ago||
Resume Driven Development, Academia Edition
Sharlin 4 days ago|||
Yep, so good that they have to be specifically reviewed because otherwise people wouldn’t believe how good they are.
Maken 4 days ago||
Actual papers are as good as ever. This is just trying to stop the flood of autogenated slop, if anything because arXiv hosting space is not free.
physarum_salad 4 days ago||
It is actually great because it shows how well it works as a system. Screening is really important to keep preprint quality high enough to then implement cool ideas like random peer review/automated reviews etc
JumpCrisscross 4 days ago||
> we are developing a whole new method to do peer review

What’s the new method?

physarum_salad 4 days ago||
I mean generally working towards changing how peer review works.

For example: https://prereview.org/en-us

Anecdotally, a lot of researchers will run their paper pdfs through an AI iteration or two during drafting which also (kinda but not really) counts as a self-review. Although that is not comparable to peer review ofc.

jruohonen 3 days ago||
A very weird move. They are now taking a stance on what science is supposed to be.

As someone commented, due to the increasing volume, we would actually need and benefit from more reviews -- with a fixed cycle preferably, and I do not mean LLM slop but SLRs. And in contrary to someone's post, it is actually nice to read things from the industry, and I would actually want that more.

And not only are they taking a stance on science but they have also this allegation:

"Please note: the review conducted at conference workshops generally does not meet the same standard of rigor of traditional peer review and is not enough to have your review article or position paper accepted to arXiv."

In fact -- and supposedly related to the peer review crisis, the situation is exactly the opposite. That is, reviews are usually today of much higher quality at specialized workshops organized by experts in a particular, often niche area.

Maybe arXiv people should visit PubPeer once in a while to see what kind of fraud is going on with conferences (i.e., not workshops and usually not review papers) and their proceedings published by all notable CS publishers? The same goes for journals.

goldenjm 3 days ago||
Their argument in favor of this change seems extremely reasonable and well-explained.
ThrowawayTestr 4 days ago||
This is hilarious. Isn't arXiv the place where everyone uploads their paper?
anthk 4 days ago||
I've seen odd stuff elsewhere, too:

https://pubmed.ncbi.nlm.nih.gov/18955255/

https://pubmed.ncbi.nlm.nih.gov/16136218/

Maken 4 days ago||
arXiv was built over a good faith assumption, where a long paper meant at least the author had put some effort behind, and a every idea deserved attention. AI generated text breaks that assumption, and anybody uploading it is not acting in good faith.

And it's a unequal arms race, in which generating endless slop is way cheaper than storing it, because slop generators are subsidised (by operating at a loss) but arXiv has to pay the full price for their hosting.

internetguy 4 days ago||
This should honestly have been implemented a long time ago. Much of academia is pressured to churn out papers month after month as academia is prioritizing volume over quality or impact.
mottiden 4 days ago||
I understand their reasoning, but it’s terrible for the CS community not being able to access pre-prints. I hope that a solution can be found.
sfpotter 4 days ago||
Please, read the title and the article carefully. That isn't what's happening.
swiftcoder 4 days ago||
It doesn't apply CS papers in general - only opinion pieces and surveys of existing papers. i.e. it only bans preprints for papers that contribute nothing new.
More comments...