Top
Best
New

Posted by secretslol 16 hours ago

What AI did to stackoverflow in a graph(data.stackexchange.com)
343 points | 401 commentspage 2
xyzsparetimexyz 15 hours ago|
Stackoverflow did it to themselves by having incredibly unhelpful users
gyan 15 hours ago||
If that was the case, the graph would never have gotten to the heights it did.

What happened is that as the corpus of useful info increased, the need to pose new Qs decreased. AI much accelerated that decline by making available an 'oracle' trained on that corpus.

liljacob 15 hours ago|||
The users being generally unhelpful wasn't an issue for them, since they were still significantly more helpful than users anywhere else on the internet. Reddit was and still is filled with completely unvetted answers (on pretty much all topics not just programming), Quora was/is a joke, Yahoo answers had some funny posts I guess but nothing you could actually learn from, what else really was there? Before AI, Stack Overflow was as good as it gets.
inigyou 15 hours ago||||
It's both. Users tolerated the hostile environment to an extent as long as the site was still the best way to get useful information. When LLMs came out, that was no longer the case.
Foobar8568 15 hours ago||||
It just accelerated the trend, and I am sure that reddit took over for a lot of new users. The different problems with SO has been well documented.

And they killed maybe one of the most side features of it : https://meta.stackoverflow.com/questions/415293/sunsetting-j...

So yeah metakill your own brands with stupid policies.

iso1631 15 hours ago|||
There was also the pattern of "closing as already answered" with an answer from 6 years earlier which wasn't actually answering the question when you dig into it. Certainly in the code stacks.
fluoridation 15 hours ago||
Definitely this. The moderators seemed to have the Lock Question button connected to their dopamine pathways.
bluedino 15 hours ago|||
That increased at the same rate of lazy/stupid users.
khalic 15 hours ago||
Seing a bell curve and singling out a factor that appears only for the 15% of the total time demonstrates some pretty extreme tunnel vision

Edit: https://postimg.cc/n9nZGLmb

embedding-shape 15 hours ago||
Yeah, I don't understand the HN title. The "downfall" seems to have began in 2018-2020 sometime, what AI was launched and popularized at that point that would have killed SO? LLMs were basically useless until GPT3 which appeared in middle-2020 sometime, after the downfall seemingly already had begun.
j_maffe 15 hours ago|||
I'd call it significant that the number of questions halved within one year following the release of ChatGPT, the biggest relative or absolute rate of decrease in the timeseries.
ipaddr 14 hours ago|||
Right around the time Google rolled out new search results removed many information based sites.
khalic 15 hours ago|||
"How AI precipitated SO's downfall" would be a correct title then
pydry 15 hours ago|||
Add it to the list:

- the downfall of junior devs

- bad hiring market

- layoffs in practically every sector

theres a ton of things where AI took credit for a trend that had already started before it started being even halfway capable.

gwerbin 13 hours ago|||
I think if you won't even admit that AI greatly accelerated these trends, you're in some kind of denial. There's no reason to believe that we would see a rapid coordinated decline in all of these things at the same time without AI, and strong reason to believe that we would see it with AI. So we have a model that makes testable predictions, and data strongly consistent with those testable predictions, in the form of an acceleration of existing downward trends. What more do you want?
pydry 11 hours ago||
>I think if you won't even admit that AI greatly accelerated these trends, you're in some kind of denial

I think if you actually look at the data for these trends rather than asking AI what it thinks you might experience some cognitive dissonance.

>There's no reason to believe that we would see a rapid coordinated decline in all of these things at the same time without AI

It's called hiked interest rates. The economy is not doing so great for several reasons but the main one is wars.

gwerbin 7 hours ago||
Hiked interest rates cause a decline in Stackoverflow activity?

My answer doesn't change. Against the background of other phenomena already causing various trends, we see acceleration of those trends consistent with a testable hypothesis about the effects of AI adoption in industry. In most fields of science that's good evidence in favor of the model.

And no I didn't ask AI about it, this is my own opinion and my own perspective.

khalic 15 hours ago|||
People love simplistic narratives, i usually don't mind but this is just ridiculous. AI hate is gently overtaking AI hype as the most stupid thing around
pydry 13 hours ago||
AI hype is still a million miles ahead and a million times dumber, especially thanks to online astroturfing.
khalic 13 hours ago||
By volume sure, a part of the anti AI crowd is pretty extreme though, death threads, bomb threads, etc.
fluoridation 15 hours ago|||
It's not really a bell curve. There was obviously a downwards trend from 2016 onwards, but 2023 definitely precipitated the fall to zero. Without AI they might have lasted at least a couple more years, or the activity might have stabilized to a new floor greater than zero.
khalic 15 hours ago|||
https://postimg.cc/n9nZGLmb

Goodness of Fit 0.911, Kurtosis -0.849, Skewness: 0.073

It's very much a bell curve

fluoridation 15 hours ago|||
Just because it's approximated by a bell curve doesn't make it a bell curve. There are quite obvious separate phenomena shaping the curve at different times.
khalic 15 hours ago|||
> just because it's approximated by a bell curve doesn't make it a bell curve

I'm going to assume this is bait...

jdlshore 14 hours ago|||
Bell curves are probability distributions. This is a time series, so it can’t be a bell curve. It just has the same shape.
khalic 14 hours ago||
Are you discarding the utility of Gaussian functions in analysis simply because the independent variable is time? A Gaussian curve can be used as a descriptive model without claiming that the observations themselves are a probability distribution

The fit does not prove causation, but it does show that the decline was already well described by a trend that began years before generative AI. If the claim is that 2023 created a separate structural break, it's different claim then the title describes

samatman 3 hours ago|||
You can't call it a bell curve unless it's from the Charles Murray region. It's just sparkling statistics.
strken 13 hours ago||||
Do you mean "just because it's a bell curve, doesn't make it a normal distribution"?
fluoridation 9 hours ago||
No, I meant what I said.
3uruiueijjj 13 hours ago|||
Those graphs look nothing alike, except for "going up and then vaguely going down."
khalic 12 hours ago||
I don't know what to say other than learn math
yorwba 15 hours ago||||
You mean the fall to a thousand questions per month. Now that the volume is low enough someone has a chance of looking at every single one of them, maybe the StackOverflow community can finally collaborate in peace, safe from the onslaught of questions that could be answered by reading the documentation.
shevy-java 15 hours ago|||
I am not sure. I think SO died way before AI and that graph seems incorrect too.

> Without AI they might have lasted at least a couple more years

Nah, their decline was already readily apparent before AI. You only need to go through old discussions and other people noticing it. AI may have accelerated the decay, but the decline happened already largely prior to AI.

marcosdumay 9 hours ago|||
Yes.

At the same time, this is graph is something that really should not look anything like a bell curve. So the format is probably just a coincidence.

Except if the "all the questions have been asked" hypothesis is correct. What I really doubt.

mmwako 15 hours ago|||
this. Thanks for pointing it out, I fell for "oh it was just AI" at first.
airstrike 15 hours ago|||
This isn't really a bell curve.
khalic 15 hours ago||
check my other response, it's very much a bell curve, statistically speaking

https://postimg.cc/n9nZGLmb

j_maffe 15 hours ago||
> statistically speaking

That's a very big word you're using there for what is basically making shapes out of clouds. A bell-curve is the amortised function of a random variable with a mean and standar deviation. What does that have to do with a timeseries dataset?

khalic 15 hours ago|||
A bell curve is not an "amortised function." Amortization applies to accounting and algorithmic time complexity, not probability distributions. You're likely thinking of a Probability Density Function (PDF). If you are going to police terminology, it helps to use the correct words. Second, fitting a curve with an R^2 of 0.911 is the exact opposite of "making shapes out of clouds.
antonvs 14 hours ago|||
> A bell-curve is the amortised function of a random variable with a mean and standard deviation.

The general notion of a bell-shaped curve is broader than that. Wikipedia has a reasonable overview: https://en.wikipedia.org/wiki/Bell-shaped_function

> “typically continuous or smooth, asymptotically approach zero for large negative/positive x, and have a single, unimodal maximum at small x.”

j_maffe 15 hours ago||
You don't just fit a Gaussian distribution to a timeseries dataset. That's not what a Gaussian curve is designed for at all. https://www.explainxkcd.com/wiki/index.php/1725:_Linear_Regr...
khalic 15 hours ago||
You are confusing a Probability Density Function (PDF) with a phenomenological curve fit. No one is claiming that time is a random variable drawn from a normal distribution.
j_maffe 15 hours ago||
> No one is claiming that time is a random variable drawn from a normal distribution.

You are doing that implicitly by fitting a Gaussian curve.

khalic 15 hours ago||
Fitting a mathematical function to a dataset does not implicitly adopt the ontological baggage of probability theory. That is a fundamental misunderstanding of applied mathematics
felooboolooomba 13 hours ago||
Peaked in 2014. That coincides with my experience. Something went wrong:

* Moderation went bad. I stopped moderating/flagging after it was deemed unhelpful?! I know it's hard to moderate a platform like that, but giving me a slap in the face when I volunteer my valuable time is not the way to do it.

* Questions closed because they weren't "programming questions", but obviously about tools devs use every day. Again and again, they were the TOP google results. You'd click on it and found a old question closed because it was considered off topic. As a business, you seriously need to ask yourself some hard questions when you fend off users like that.

nolok 13 hours ago|
Yeah, I used to mostly answer there instead of ask and was in the top % for several big tags whatever that means / is worth, but then at some point I realized things had changed and I was spending more time fighting rules and guidelines rather than sharing knowledge and having a good time so I just stopped.
arjie 7 hours ago||
They didn’t stand a chance. Even in the era where they made a great community etc etc AI would have destroyed them. Other people are just a poor source of information now that we have dramatically good search across all human knowledge.
JustRouzbeh 12 hours ago||
It’s pretty sad though. Like many developers, I used Stack Overflow a lot when I was starting out, and it helped me solve countless early programming problems

A lot of what we have today was built with help from that community

kittoes 12 hours ago|
The community did themselves no favors. I personally don't have any issues whatsoever dealing with it, but the overwhelming majority of my coworkers over the past decade haven't ever asked a single question. They saw how others were treated or heard about horror stories from some of the few souls who made the attempt and went "Why bother?"
zzril 12 hours ago||
I have never actively asked a question on SO either. But for every programming question I ever had (literally!), I always either found the solution to it, or the reason why my question was stupid. (Which usually boils down to me not understanding sthg about the technology I'm working with.)

Digging my way through old SO posts has tought me so much... but now, it's AI time and I find myself pasting my questions into a prompt most of the time, rather of thinking about what the correct keywords to google would be. Which, in a way, is faster, but at the same time I now feel like I'm not learning anything new anymore...

pluc 15 hours ago||
So... nothing that it wasn't already doing to itself? There's no one drop where "AI got into the market", SO had been declining steadily for years. I actually expected this post to be about how SO survived by selling its internal organs to AI.

Now do a graph for the money.

https://www.wired.com/story/google-deal-stackoverflow-ai-gia...

j_maffe 15 hours ago|
> There's no one drop where "AI got into the market"

Sure there is, number of questions halved from 100K in first of November 2022 to 50K exactly one year later.

sota_pop 7 hours ago||
I recall some moves where SO announced they would be revising ToS to disclose their intent to repackage and use the site’s content for sale and/or model training.

Contributors responded by going to delete their own respective contributions en masse. Upon doing so, were banned by the platform mid-process which then led to people going back to revise their contributions to be false rather than deleted.

I guess that’s “LLM related”

Kuinox 15 hours ago||
The stackoverflow moderation is the reason I do not post on it. You have middle party with no competence on the technology trying to do useless moderation.

Instead I directly go on the project github page and ask the question directly to the mainteners.

robryan 14 hours ago||
Some of the pre-ai decline in questions might just be that they had filled out of alot of the question space. What might be more interesting is the traffic graph as it would be possible to have a decline in questions but still have traffic rising to the existing ones.

Today I'd expect even when someone is googling and a question is relevant that most people will just read the AI overview.

hazrmard 7 hours ago|
Does the peak starting in mid-2020 correspond to mass speculative / adversarial / bandwagon hiring by tech companies during the pandemic?
More comments...