Top
Best
New

Posted by rndsignals 13 hours ago

Medical students are using popular research tool to pump out misleading studies(www.science.org)
119 points | 68 comments
bigmattystyles 5 minutes ago|
I’m currently facing a severe health condition and I can’t help but ask various LLMs about it. They all will eventually offer solutions or avenues that sound promising or ‘easy’ when I know better, that the path ahead of me is hard, however they will non-chalantly offer a path forward that it will insist will work. The best part is when it cites commercial websites promotional statements as facts, though it will also misinterpret medical journals if I say to restrict itself to that.

In closing, my Redfin escapism has shifted to LLM medical escapism, I know better but if you don’t or you are in even more dire straits, it provides such an illusion of hope and that’s dangerous.

zemvpferreira 3 hours ago||
As a former scientist it's embarrassing how easily the dismal state of science today could have been predicted decades before by applying Goodhart's Law, or any simple train of thought on incentives and moral hazard. Instead we chose to assume scientists collectively behave on a higher plane. No wonder the general public distrusts "the intelectual elite", we deserved it.
tstactplsignore 1 hour ago||
This article is about MD doctors, not scientists (although there may be some MD-PhDs in it, and yes MDs can get research positions).

MD doctors poll at extraordinarily high levels of trust, over almost any other professional group in the United States. So it really isn't correct to directly link this article's topic to "distrust". The effect you're talking about may exist in science, but this article is essentially a counter example to the effect you propose: clinicians publishing bullshit, but retain a high level of public trust.

Especially because the article is basically entirely quoting practicing scientists who identified this problem in the first place! More real scientific training or collaborating for clinicians who want to (or have to) do research could potentially improve the situation.

fluidcruft 47 minutes ago||
MD doctors (and even to a large extent DO doctors nowadays) are philosophically grounded on science. An MD fundamentally practices science. That used to be one of the key differences between MD and DO physicians, but science has been so successful at advancing the standard of care that DOs cannot ignore it anymore. That's just to say that MDs are expected to be fluent in science and it's not some arbitrary expectation of bureaucracy run amok.
georgeburdell 8 minutes ago||
Due to the high trust they get from the public, I have run into many MDs who are used to making unfounded assertions with little pushback.
jjulius 1 hour ago|||
>Instead we chose to assume scientists collectively behave on a higher plane.

People are people.

mmooss 6 minutes ago||
If the solutions are so simple, what are they? Yes, incentives are distorted, but what system - that does not rely on everyone being on a higher plane - would have better incentives?

By your definition, every human endeavor is dismal and always has been - all are corrupted and flawed to some degree. Is there evidence that current science is more dismal than others or than before? You can look at any day in history and see people saying the same things about how it's so dismal and not like the good old days.

> No wonder the general public distrusts "the intelectual elite", we deserved it.

The general public has no idea about scientific publishing, publish or perish, or the distorted incentives it creates. Science has delivered at an incredible level for centuries, arguably more than any other human enterprise. Covid-19 vaccines were available in record time - it wasn't the science that caused it to go somewhat off the rails.

currymj 1 hour ago||
residencies have decided to outsource part of their hiring decisions to journal peer-review processes. so now for some submissions, editors and reviewers are not actually doing scientific peer review, but rather screening job candidates for hospitals.

peer review is built to assume good faith work by people who are all part of a community of scholarship, it can partially hold up to people within the community gaming metrics. if people are just going to appear, game the system to publish some papers, and then disappear into their real careers, there's no hope of this working.

i don't understand why residencies want med students to publish papers anyway. it's very difficult to do good scientific research, it requires training, time, and almost always apprenticeship. none of this is part of the medical school curriculum, which is why we need special MD-PhD programs for people who want to do both. nobody expects that doing a PhD in biology or epidemiology would give you any clinical know-how, why is it reasonable to expect the reverse?

gowld 1 hour ago|
Residency is a guild (aka, anticompetitive trust) designed to limit the medical workforce and drive up doctor salaries. It needs artificial metrics to ratonalize the increasing salaries as being for "increasing talent".
whizzter 6 hours ago||
Researcher/academics pay/promotoins should be contingent on reviewing,challenging and reproducing papers rather than publishing quantity, because publishing cartels and AI has already degraded most research fields.
pjc50 5 hours ago||
Should be, but you've got to tell the funders that.
whizzter 2 hours ago||
I think I was more referring to academia than commercial research even if there is a large intersection.
pjc50 2 hours ago||
Academia is almost entirely steered by government policy, at least in the Anglosphere. There is a bureaucratic not-very-scientific evaluation process for research proposals. And things like the https://en.wikipedia.org/wiki/Research_Excellence_Framework , which has been criticized for practically requiring people to game the "impact" metric in order to keep their jobs.
ktallett 6 hours ago|||
Reproducibility in many scientific as areas has been made almost impossible. We have got to the stage where IP matters more than scientific rigour so methodology is purposely left out.
functionmouse 3 hours ago||
so... empiricism is over?
whizzter 2 hours ago||
I wouldn't say so, more that lawyers and capitalistic interests goes before scientific advancements, that doesn't preclude independant (or guided) re-discovery and eventual replication of findings.
humanfromearth9 5 hours ago||
No. You can't spend all your money on rehashing past results. Some, OK, all, not. In many fields, the money is needed for discovery.
7734128 5 hours ago|||
Discovery is quite worthless if the discovery can't be trusted enough to continue building upon.
whizzter 2 hours ago||||
I'm not entirely disagreeing, right now however there is so much fraud that when it intersects with things of interest causes millions(billions?) to be spent on chasing the wrong leads (see findings on that 2006 Amyloid Plaque paper regarding Alzheimers research).

I'm mostly saying that being reproducible should become a higher badge of quality, right now reviewers in cartels can boost a researchers credibility by accepting each others articles to papers to let them become "influential" and money is then redirected even more to bullshit research (ie pure waste).

If up to 50% of research grants is spent on bullshit research based on fraud, spending 10% by earmarking it for reproduction to weed out irreproducible fraud is money well spent.

redsocksfan45 2 hours ago|||
[dead]
eunos 2 hours ago||
That aside I am a bit perplexed that almost absolute insistence of medical students to become researchers as well, it seems just become a pure practitioner is not a feasible option. To make it worse it looks more acceptable that a doctor really provide bad service or lacking communication and empathy to patients than not being a researcher.
nickgros 1 hour ago||
Guidelines and standards can change quickly. It's important for clinicians to have a foundational understanding of statistical methods and to be able to critique studies.

For certain specialties, the number of residency positions is so limited that medical students have to publish research to be competitive, even if they have no interest in doing so later in their career.

But altogether I sort of agree, the incentives are pretty maligned such that for many it's just easier to become a bad scientist with more publications than a good one with fewer.

gowld 1 hour ago||
Publishing a study does not show "a foundational understanding of statistical methods and to be able to critique studies." It's literally the opposite!
GerryAdamsSF 1 hour ago|||
Most doctors drop the research once they get a residency spot. It's just the selection mechanism for residency.
alcasa 2 hours ago|||
Clinical research is in a weird spot, where you need both clinical experience and research experience. Getting the former already requires long hours and you are swamped with work. The latter is highly age gated and if you want to pursue research you often need to achieve specific milestones before a certain age, e.g. to be considered for tenure etc.
redsocksfan45 2 hours ago||
[dead]
bagelbob432 1 hour ago||
Some context to this is recently the big "step 1" board exam that covered the basic science portion of medicine went from being graded to being pass/fail. Since then, all the medical students where I work have started trying to get any kind of research on their CV to stand out during their residency application. New 120 participant "conferences" where med students can present posters are popping up, more med students trying to get into the labs for rotations than ever before. A lot of it is really low quality, but for them, if they don't have any research, and someone else does, they will be ranked lower and get worse jobs. Weird effect of whatever happened to make Step 1 pass/fail
fn-mote 11 hours ago||
My assumption is the credibility of a non-PhD-holding medical student’s research is 0, just like (almost) any other inexperienced researcher.
thomasfedb 9 hours ago||
As a clinician-academic who published in The Lancet during medical school, I think this goes a bit far. Unfortunately student doctors are encouraged to publish whether or not they actually have an interest in research… but that shouldn’t discount the work of those who are genuinely engaged.

But certainly we should always approach the literature critically, including the author list, journal of publication and its peer-review practices, and the methods.

tremon 4 hours ago|||
> student doctors are encouraged to publish whether or not they actually have an interest in research… but that shouldn’t discount the work of those who are genuinely engaged.

How do you propose the interested public make the distinction between genuine engagement and forced encouragement? Isn't it the task of journals to make that distinction before publishing? I don't think you can fault the public for dismissing everything out of hand when both academia and the journals are actively turning scientific publishing into a market for lemons.

gowld 1 hour ago||||
Were you first-author on an unsupervised project and paper, or did someone stake their reputation on your work?
gbnwl 5 hours ago||||
As usual HN posters are hyper aware of other's credentials while ignoring that their BS in CS (if that) doesn't magically qualify them to assess everything in every domain.

"I'm a software engineer, I'm sure if I had the time to study Neuroscience, I'd figure out what all of these researchers failed to realize all these decades! I (alone) have the magic of critical and logical thinking"

Avicebron 3 hours ago||
A lot of us here have masters, PhDs, have published in academia, worked in the hard sciences or different engineerinf disciplines.

But I agree, when youre on the internet no ones knows you're a dog.

BrtByte 7 hours ago||||
I think this is the right distinction
bflesch 8 hours ago|||
I was severely disillusioned about the quality of clinical studies.

Would you publish if the head honcho of your double-blind study insists to know what treatment a certain patient is receiving?

You have this discussion about research ethics and subsequent beratement once, and then you either mentally check out or go to another hospital.

niekmaas 8 hours ago|||
Well, that is a statement..! As an MD PhD with over 60 (co-)publications including multiple in top 1% journals I can say for sure that this is untrue. Of course this may be different per topic and country, but there is perfect research being published by non-PhD scientists. In fact, the PI from a top-tier US university I collaborate with for over 10 years doesn't even have a PhD.
11101010010001 1 hour ago|||
Both your statement and the OPs are an appeal to expertise which is not a substitute for the scientific method.
KeplerBoy 8 hours ago|||
You can be a PI without having a PhD?
internet_user 6 hours ago||
Even Hassabis found time to do a PhD. This is extra strange.
sebmellen 9 hours ago|||
This is really far too broad a brush.

Do most medical students publish useless case studies trying to jockey for residency spots and signal hustle/devotion? No doubt!

But there are a good handful of medical students who are still (surprisingly) in it for the medicine and not the money. And that handful is exceedingly capable; no reason they can’t publish valuable work with the right collaborators and resources.

myroon5 9 hours ago|||
> no reason they can’t publish valuable work with the right collaborators

Despite h-index claiming to balance quantity and quality, it obviously incentives quantity over quality (no single publication can increment h-index as much as churning out a few worthless publications that cite each other); med students overwhelmingly follow those incentives trying to secure better residencies

mishellaneous 7 hours ago|||
in that case, it's a question of proportion. we cannot automatically conclude that a (supposed) "good handful" doing good research makes up for "most students" doing bad research.
BrtByte 7 hours ago|||
A med student can absolutely contribute useful work, especially with good supervision. The issue is more that inexperienced authors plus publication pressure plus easy tooling is a bad combination
samuraijack 7 hours ago|||
LLVM was a masters thesis project (not medicine related but research by non PhDs should not be disregarded imo)
philipallstar 6 hours ago||
LLVM was a thing that demonstrated its value by working when you used it. And you can't judge a population (all non-PhD theses) by its tippety-top performers, particularly when there are poor incentives involved.
mishellaneous 7 hours ago|||
a friend of a friend who did a stint in biomedical academia told me that the researchers in their field did not hold research coming from the medicine community in high regard
boelboel 5 hours ago|||
Knew a professor statistics from a world renowned institution. He worked in nephrology for 10-20 years and would tell many stories about the worst practices he's seen and researchers pushed him to allow.

Medicine was among the worst if not the worst according to him. Didn't really want much to do with it anymore. Basically a case of subpar statistical knowledge and bad incentives.

gowld 56 minutes ago|||
How can you tell when you are reading a paper?
aardvark92 10 hours ago|||
I guess it depends on who the coauthors and PI are - some academic mentors can be overly trusting and ‘hands-off.’ A lone medical student’s self published paper shouldn’t be worth much though…
bflesch 8 hours ago|||
In the end it is about personal integrity and idealism, no matter what the titles are.

Totally different if someone's self image is that of a researcher for benefit of humankind or if they pick the career because they want to drive a Porsche.

elendilm 6 hours ago|||
What has PhD got to do with anything. Research is research regardless of who does it if using proper scientific method.

Such obvious common sense appears not obvious after all.

NotGMan 9 hours ago|||
Since we have seen that 50%+ of findings even in medical and other natural sciences are not repruductible it's obvious that even PhD people are mostly incompetent.
harvey9 2 hours ago||
Some of the examples of bad design mentioned in the article are quite shocking, if we are assuming 'medical student' is not some 18 year old rookie but a person who already has several years of university level study behind them.
spchampion2 1 hour ago|
My potentially unpopular opinion: Congress should ban residency programs from using Medicare training dollars to pay for research. They should do this with the goal of speeding up the training pipeline for actual practitioners, many of whom are now required to spend a year of their residency doing research of some kind.

If medical residents, or teaching hospitals, want people to do research, they should go get funding from established research funding sources that have standards and practices for funding and monitoring research.

GerryAdamsSF 1 hour ago||
This is wrong.

It's medical students, not residents, who take research years, and that's only for extremely competitive specialties.

The lack of doctors, as it has always been, is because of the shortage of residency spots.

That cannot be addressed without Congress reversing 50+ years of neoliberalism trends and political failure and refusal to invest in public services and/or a communist revolution. Good luck fixing that.

gowld 55 minutes ago||
Congress should end the AMA guild's chokehold on linking "Medicare" to medical training at all. It's a complete grift.
More comments...