Posted by timr 1/25/2026
Benefits we can get from collective works, including scientific endeavors, are indefinitely large, as in far more important than what can be held in the head of any individual.
Incitives are just irrelevant as far as global social good is concerned.
And therein lies the uncomfortable truth: Collaborative opportunities take priority over veracity in publications every time.
Of course doing so is not free and it takes time. A paper represents at least months of work in data collection, analysis, writing, and editing though. A tarball seems like a relatively small amount of effort to provide an huge increase in confidence for the result.
IMHO this should be expected for any, literally any publication. If you have secrets, or proprietary information, fine - but then, you don't get to publish.
That's not right; retractions should only be for research misconduct cases. It is a problem with the article's recommendations too. Even if a correction is published that the results may not hold, the article should stay where it is.
But I agree with the point about replications, which are much needed. That was also the best part in the article, i.e. "stop citing single studies as definitive".
I read the paper as well. My background is mathematics and statistics and the data was quite frankly synthesised.
But the article is generally weird or even harmful too. Going to social media with these things and all; we have enough of that "pretty" stuff already.
However there are two problems with it. Firstly it's a step towards gamification and having tried that model in a fintech on reputation scoring, it was a bit of a disaster. Secondarily, very few studies are replicated in the first place unless there is a demand for linked research to replicate it before building on it.
There are also entire fields which are mostly populated by bullshit generators. And they actively avoid replication studies. Certain branches of psychology are rather interesting in that space.
Maybe, I cannot say, but what I can say is that CS is in the midst of a huge replication crisis because LLM research cannot be replicated by definition. So I'd perhaps tone down the claims about other fields.
Pushing for retraction just like that and going off to private sector is…idk it’s a decision.
She was just done with it then and a pharma company said "hey you fed up with this shit and like money?" and she was and does.
edit: as per the other comment, my background is mathematics and statistics after engineering. I went into software but still have connections back to academia which I left many years ago because it was a political mess more than anything. Oh and I also like money.
Citation studies are problematic and can and their use should be criticized. But this here is just warm air build on a fundamental misunderstanding of how to measure and interpret citation data.
This is a frustrating aspect of studies. You have to contact the authors for full datasets. I can see why it would not be possible to publish them in the past due to limited space in printed publications. In today's world though every paper should be required to have their full datasets published to a website for others to have access to in order to verify and replicate.
We need to throw all of this out by default. From public policy to courtrooms, we need to treat it like any other eyewitness claim. We shouldn't beleive anything unless it has strong arguments or data backing it. For science, we need the scientific method applied with skeptical review and/or replication. Our tools, like statistical methods and programs, must be vetted.
Like with logic, we shouldn't allow them to go beyond what's proven in this way. So, only the vetted claims are allowed as building blocks (premises) in newly-vetted work. The premises must be used how they were used before. If not, they are re-checked for the new circumstances. Then, the conclusions are stated with their preconditions and limitations to only he applied that way.
I imagine many non-scientists and taxpayers assumed what I described is how all these "scientific facts" and "consensus" vlaims were done. The opposite was true in most cases. So, we need to not onoy redo it but apply scientific method to the institutions themselves assessing their reliability. If they don't get reliable, they loose their funding and quickly.
(Note: There are groups in many fields doing real research and experimental science. We should highlight them as exemplars. Maybe let them take the lead in consulting for how to fix these problems.)
> We need to throw all of this out by default. From public policy to courtrooms, we need to treat it like any other eyewitness claim.
If you can't trust eyewitness claims, if you can't trust video or photographic or audio evidence, then how does one Find Truth? Nobody really seems to have a solid answer to this.
Next, we need to understand why that is, which should be trusted, and which can't be. Also, what methods to use in what contexts. We need to develop education for people about how humanity actually works. We can improve steadily over time.
On my end, I've been collecting resources that might be helpful. That includes Christ-centered theology with real-world application, philosophies of knowledge with guides on each one, differences between real vs organized science, biological impact on these, dealing with media bias (eg AllSides), worldview analyses, critical thinking (logic), statistical analyses (esp error spotting), writing correct code, and so on.
One day, I might try to put it together into a series that equips people to navigate all of this stuff. For right now, I'm using it as a refresher to improve my own abilities ahead of entering the Data Science field.
Scientists that have studied this over long periods of times and diverse population groups?
I've done this firsthand - remembered an event a particular way only to see video (in the old days, before easy video editing) and find out it... didn't quite happen as I remembered.
That's because human beings aren't video recorders. We're encoding emotions into sensor data, and get blinded by things like Weapon Focus and Selective Attention.
Much of what many learned about life came from their parents. That included lots of foundational knowledge that was either true or worked well enough.
You learned a ton in school from textbooks that you didn't personally verify.
You learned lots from media, online experts, etc. Much of which you couldn't verify.
In each case, they are making eyewitness claims that are a mix of first-hand and hearsay. Many books or journals report others' claims. So, even most education involves tons of hearsay claims.
So, how do scientists raised, educated, and informed by eyewitness claims write reports saying eyewitness testimony isn't reliable? How do scientists educated by tons of hearsay not believe eyewitness testimony is trustworthy?
Or did they personally do the scientific method on every claim, technique, machine, circuit, etc they ever considered using? And make all of it from first principles and raw materials? Did they never believe another person's claims?
Also, "scientists that have studied this over long periods of times and diverse population groups" is itself an eyewitness claim and hearsay if you want us to take your word for it. If we look up the studies, we're believing their eyewitness claims on faith while we've validated your claim that theirs exist.
It's clear most people have no idea how much they act on faith in others' word, even those scientists who claim to refute the value of it.
This one is pretty egregious.
I only needed the Spanish translation. Now I am proficient in spoken and written Spanish, and I can perfectly understand what is said, and yet I still ran the English through Google Translate and printed it out without really checking through it.
I got to the podium and there was a line where I said "electricity is in the air" (a metaphor, obviously) and the Spanish translation said "electricidad no está en el aire" and I was able to correct that on-the-fly, but I was pissed at Translate, and I badmouthed it for months. And sure, it was my fault for not proofing and vetting the entire output, but come on!
Actually it’s not science at all.