Posted by geoffbp 15 hours ago
I am interested in if it's a fundamental limitation, ie models that have gaps that need to be corrected are going to be bad at recognizing those gaps, and leaving instructions to fix them. Or if models just aren't trained on this
It seems akin to my experience with coding agents, where models are good at implementation but not great at abstraction
Firstly LLMs need to have more training on speaking to several different people.
Humans have a lot of training on interacting with many different individuals and LLMs feel extremely one thread chat heavy. Like if you read the entire thread the LLM langauge is usually consistent but if you send someone a partial artifact it feels disconnected in a broken way.
This includes their code changes and doc changes which inheret this lack of audience.
For the more general idea of keeping good artifacts for themselves I think an interesting idea would be to look into context language models instead of autoregressive ones.
https://arxiv.org/abs/2609.37725
Context language models need to manage their own context to complete a task. (They see their own context as a file).
This naturally forces this innate ability of managing relevance.
The article explicitly excludes this from consideration early on though:
By “agent memory,” we mean knowledge that Claude gains from experience...saved as context it can read later. It excludes other ways models can learn or manage context, such as changes to their weights
Those seem fundamentally different processes, and adjusting weights does seem a much better idea after all, but then in that world everyone has a different AI and how do you apply updates? For a company like Anthropic this is of course a big problem and that is probably why they don't like that route. Could we store weight deltas or something similar and reapply them over a different model?
I wonder if we are so advanced as we think that these constructs (dreaming), are the right ones for improving intelligence in a program that does not need to be modelled after us or constrained by what evolution gave us.
Part of me feels the pang of regret and shame seeing so many little details I researched and dreamed[1] (ha!) of prototyping ~2.5y ago expressed and validated so cleanly. Obviously A) I never did get the prototype working nor publish the underlying ideas and B) this blog post (paper? ...plog post?) is building on it at a scale and with quality I could never dream of (fig. 3 is especially striking), not to mention all the new ideas mixed in. Still... sharing cause it helps, I think, and because I expect y'all can relate; if you're old enough, it's very similar to the feeling you get when you think about how much Apple stock you could've bought in the 90s.
Anyway, more materially:
1. Pretty interesting to see Opus 5.5, Sonnet 5.5, and Opus 5 tied on the basic memory test (aforementioned fig. 3); the Opus comparison alone seems to imply either a hard plateau or that 5.5's huge improvements were accomplished by making good practices more likely, but the Sonnet detail throws that for a loop... I suppose the plateau explanation isn't contradicted, so maybe that's right (for this benchmark).
2. Fig. 4 is gonna do rounds on Reddit and Twitter, my god. Pretty stellar, intuitive depiction of the Mythos step-change that enabled true agentic engineering for the first time this summer! The highlighting is also just the right amount of facetious; love it.
3. Pretty bummed to see no followup philosophy references. We know you have them, Anthropic nerds! Do not be ashamed of the soft and slippery.
4. I am now reminded that I knew they were trying dreaming at scale, as I'm actually on the waitlist for it! And that backs up my overall takeway, actually: I can't help but feel like they're either A) vastly underestimating the amount of CogSci literature that could help them go beyond the "dreaming is when you think in your offtime" paradigm, or B) not sharing their real learnings.
Today's RSI announcement and the final sentence of this seminal plog post being a pitch for enterprise customers supports the latter, I think. That's ostensibly a good thing from a societal stability perspective, but I can't help but feel left out -- not to mention paralyzed by fear of the known unknown...
That "I never got to finish" feeling is the absolute worst. I've been struggling with it for years, and I feel like understand ya here.
It absolutely sucks you feel that, and I hope you find some comfort soon, however it may come.
If you're looking for a "so what now?" Moment, there's only one thing I can say that I've been proselytizing a bit: share everything. All the time. And get people to look at it and be your hype person; and if you're not getting that in your friend ground or workplace or community, find a new one to add to.
The only way I've found to beat that feeling of "why is it i have the good ideas years ahead but can't do anything with it" is to roll more dice. That's it. These larger companies just essentially have infinitely many die and they're rolling them infinitely faster.
You've done nothing wrong. You're just with some other folks like us that don't have the same die rolling power.
Don't give up on your ideas. Don't stop experimenting. Make your own tools because it's for you, and maybe your dice will come up that you get the other person in the world that sees it and goes "yes!" And you get all the fancy posts and stars and attention and such. Because that feeling of validation - not just popularity, but acceptance and being seen and heard for one's craft - is one everyone seeks and needs in life. It's not a weakness. It's being a human.
Thank you for sharing your experience with internet strangers.
Many of us have ideas, ideas are cheap. Executing a complex idea well from thought to completion is much more rare.
So my thought about sharing and not thinking about "my secret sauce" anymore is just necessary. Share what you think your cool flavor of an idea is, someone else can groove on it, and maybe you and them will grow it into something more.
A major problem as I see it is kinda geek-class-warfare coming to a head. So much of my personal and anecdotally collected experience from others is essentially being locked out of knowledge and ideation because a "upper geek class" decided that they were not worthy of having their idea shared, specifically because that class thought they could make money from it, or because they were insular for some other in group reason. That meant good ideas that just needed a single word or connecting bridge from one topic to another ended up withering, and mediocre ideas ended up being accelerated because of just sheer populace force.
And that just makes people feel bad. Someone working for 2 1/2 years on an idea that just happens to come out on the Internet is not going to feel good. It's going to make them feel invalidated, it's going to make them feel exhausted, it's going to make them feel like they didn't do enough. When it's just not the case.
Anyway. I just wanted to say something to this as well, because it's sort of just dismisses a person's ideas. I don't think you're intentionally doing that, but even repeating the phrase, "ideas are cheap" cheapens what it actually takes to come up with an idea. And now in 2026, when literally any idea can become something real, I really don't think ideas being cheap means ideas aren't valuable.
> And that just makes people feel bad. Someone working for 2 1/2 years on an idea that just happens to come out on the Internet is not going to feel good
This was more than an idea for you and more than a hobbey at least according to my definitions of those terms. I'd call it a passion research / early stage business venture depending on how much effort you put into it and what your intentions for it were.
The (penti-)closing statement is also pretty damn poignant, and very true. I daresay your thoughts on publishing deserve to be published!
Substantively, I absolutely agree in theory -- I even recently set up my Substack with this in mind, despite that being an insane thing to do when frantically job searching! I (/Claude...) just found a brief exchange[1] I had w/ @simonw on this very topic almost exactly two years ago below a heartfelt blog post[2] about someone's personal reasons for struggling with this, even. Reading it now mostly serves to remind me of the roadblocks, though:
1. I was quibbling with his advice to "force" yourself to share by reflex (and the author's framing of the failure to do so as "laziness"), and two more years of reading about, discussing, and managing AD[H]D symptoms have only strengthened my convictions there. Charitably, getting into the habit of publishing is an investment, and my metaconscious/'inner critic' has plenty of quite plausible arguments as to why it wouldn't be worth the squeeze[3]; less charitably, I'd note that the relatable blogger in question only posted twice over the following 12 months.
2. That exchange took place just as A) reasoning models were changing everything, B) artificial articles outnumbered human ones for the first time.[4] In the intervening years we've seen AI turn from niche to vitriolic, and at least a dozen variations of "If you didn't care enough to write it yourself, I don't care enough to read it" posts hit the front page of HN. Obviously we all agree and are talking about human writing/ideating here, but the overall vibe is still undeniably... tense, let's say[5].
I still appreciate your comment and agree, mostly because the anxieties presumably lessen considerably with exposure and practice. In other words, I'd get used to it.
In fact, I just queried the corpus that I ironically have yet to publish the pipeline for, and I've written 6,280 real comments (>50 words of original text) and 1,788 long ones (>100 words) since that day on Reddit, Bluesky, and here. Some of that might be real differences --say, ostensible anonymity on Reddit and lower expectations for mere comments-- but the only factor that accounts for a spread of 6,280-0 is habit and comfort, I think.
As usual (55 times since November 11th, 2024): apologies for the rant, y'all.
---
[1]: https://news.ycombinator.com/item?id=42101327#42104145
[2]: https://swapnilchauhan.com/blog/procrastination-and-the-fear...
[3]: Especially now that we're at the tail end of the initial six months of agentic engineering; investing the emotional energy to share my work is a lot harder to justify when I could orchestrate more+better engineering work in those same few hours than I could've accomplished in a few days two years ago.
[4]: https://graphite.io/research/ai-now-writes-as-many-online-ar...
[5]: Wow, I was more right than I thought! Claude just found a preprint showing that "pejorative accusations" more than doubled in the final third of 2024 --jumping from 7.9% in August to 17.9% in December-- despite poor accuracy: https://arxiv.org/abs/2606.12073
Another example of this is "skills", which has a lot of branding for what are essentially markdown files
This is an interesting article, but IMO what makes this interesting is not the idea, but the empirical evidence that you are probably equally capable of putting together if you had millions of dollars
That’s not accurate. One leading hypothesis is that the experience of dreaming is a side effect of memory processing and consolidation[1]. Using the name of a side effect to describe the process that produces it is misleading, and at best it goes well beyond what’s actually known.
[Edit: and the side effect in question, dreaming, is closely tied to consciousness in humans. Using an “analogy” with those connotations is an unfortunate mistake.]
[1] “The process of memory reactivation and consolidation in the sleeping brain appears to influence conscious experience during sleep, contributing to dream content recalled on awakening.” — https://pmc.ncbi.nlm.nih.gov/articles/PMC4704085/
I wouldn’t be particularly surprised if Anthropic were to reconstitute itself as a religion.