Posted by gslin 3 hours ago
Friends, look at the prompts that Anthropic's own people are putting into the machine:
> A few hours after the first message, we found that Claude was still searching for simple attacks and sent a message: “no again the goal is that we have highly inteligent [sic] model as good top researcher, we want to find new attacks”;
> The next morning, Claude wanted to try to change the target to a different cipher; we reminded the model: “no we don't want to change the targets [...] agian [sic] we need to find something that worth [sic] publishing”;
> That night, we sent one final message offering words of encouragement: “again we are not looking for low hanging fruit, we want proper research to find genuinly [sic] hard findings.”
All of that RLHF and fine-tuning effort is going toward making prompts like this, or worse, work with no fuss.
Skills, CLAUDE.md/AGENTS.md should only ever be used if the model struggle at something or doesn't know how to use something. Vast majority of project should never need a skill or CLAUDE.md. If you writing React apps you don't need these.
Give a LLM a bash tool and a prompt and it will outperform your complex setup with skills and tools.
Or what would make automatic doors work like on Star Trek and not in real life.
The answer is: the system must obviously see much more than your prompt. It must have continuous awareness of you and what you're doing, so it can understand intent behind your short request (or action, like approaching the doors vs. passing by them) and "do what you mean" instead of act like regular computers today.
1. I'm glad the second kind works too;
2. First kind is where I find my overall throughput to be literally constrained by my typing speed;
3. Most importantly: those prompts you quote aren't just "half-assed" like sibling comment states; they're different. The style of writing, and the typos, capture emotional valence. It's a signal.
Again, I too produce such prompts - including the exact same typos - when under pressure and irritated by the direction the model is taking.
A little odd at first but absolutely amazing for the purpose of piling context into an LLM.
Can't you type faster than you speak? Doesn't your speaking inhibit your thinking? Aren't you self-conscious talking out loud? How are our experiences so different?
The key thing with these voice systems is that you do not need to edit anything. You can literally just stream of consciousness into them, no editing, include the backtracking, the live-revisions, etc., and it will actually all produce vastly better context for the LLM than the written thing you took even 30 seconds to edit for clarity or brevity.
I had a very similar disposition towards this idea just 6 months ago. I highly recommend trying it out. The key thing is that you do not need to edit. Just keep talking. Try it for a few weeks!
It's a skill like any other. You start out stuttering and second-guessing yourself, but after a while, you get better at it. And the LLM smooths out the odd mistakes better than you might think.
My limiting factor is that 99% of the day I'm around people - either at work, or at home with wife and kids. There's almost no point during the day I could feel comfortable talking at an AI, and even if I stay up late, then talking risks waking the kids up.
Can't wait for some kind of subvocalization microphones to become a thing.
I have zero desire to talk to an ai though, that was cool for about 20 minutes on my pentium 1 acer computer. Hasn’t been since. Old competent non paid Alexa was good for timers as well, the rest of the platforms a turd, nice timers though.
Oh back in the days, i.e. 20 years ago, I had a better voice control system than anything afforded by Alexa or Apple or others, using MS Speech API in its custom constrained grammar mode, plus some bootleg samples of Star Trek's computer voice + a sub-dollar microphone soldered to a long cable and hung on the side of the wardrobe.
The trick that made it work? Microsoft Speech API actually let you train voice to your own text corpus. I'd prepare all combinations of commands I want to issue, print it out, and train it over a dozen short sessions in several locations of the room and at different ambient noise levels (from silent through various genres of music playing at various loudness). End result was more reliable and had better voice-mismatch rejection than any current system I've tried.
Oh, and the real kicker? This all worked fully locally; this was before cloud was even a thing. Turns out you don't actually need cloud for reliable voice control. Nor that much processing power; PC I had then was relatively budget even for 2006.
> Importantly, this is just one of many (autonomous) sessions where Claude worked on discovering new ideas. Many sessions resulted in no new discoveries; other follow-up sessions improved on the insight developed in this one. This document was produced by having Claude rewrite the chain of thought to include more detail to make it easier to read.
Similarly, when effort is applied to an open problem, such as the Riemann hypothesis or P v NP, without progress, it "hardens" the problem: it makes the problem feel more daunting to whoever takes a stab at it next.
Andrew Wiles, whose interview also hit the homepage today (https://news.ycombinator.com/item?id=49075264), couldn't just tackle Fermat's Last Theorem head on, he had to wait until a different, modern problem reduced to it, because FLT had gathered this mystique of unassailability through its 300 years of existence.
A thing I worry about is that as AI transmutes tokens into effort, it'll split the world into two: some problems will yield, making human effort entirely unnecessary, and others will harden to the point where human effort will feel increasingly less worthwhile, because "even AI couldn't solve it". I don't like this. AI is spiky, so I suspect it'll continue having major blind spots, and yet its mere presence will probably have a chilling effect on what would have otherwise been useful human effort.
And
“Over the course of a week, one Anthropic researcher worked together with Claude to develop the HAWK attack, and another researcher built a scaffold4 that allowed Claude to fully autonomously discover the AES attack.”
Spending $100k in tokens in a week is an impressive feat even with massive parallelization. I suspect the TPS their internal folks have access to is far higher than their bulk public endpoints.
There’s a tech aristocracy rapidly emerging in our society and it’s going to tear us apart.
I predict the same will happen with AI: certainly the latest and greatest will still command a steep price (yes, supercomputers are still a thing) but for most people who just need something reasonably fast and powerful, cheap (or free) AI will do the trick, especially when run locally.
So no, the aristocracy won't have a lock on the technology because tech is always being democratized. Until arbitrary computation itself is outlawed (and yes, I know, governments and industry are always inching us closer to that), we'll be ok.
Why have all the mathematical (and now cryptographic) breakthroughs come from OpenAI and Anthropic?
Is it possibly because the Chinese models are so benchmaxxed they can't make novel discoveries?
"The attacks described in these two papers are the strongest attacks we have found to date. We are sharing them after a period of consultation with US government and industry leaders. But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact. We believe answering this question will require input from academia, government, and industry. We hope that our work here will help launch these conversations."
And a veiled pitch to real cryptanalysis researchers: "Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim"
this is pretty interesting. the way it is written doesn't make it sound like the collaboration actually led to the discovery, but rather just the stochastic nature of each thread in the search. it would be interesting to replay and repeat the search (possibly with prior/context pertubations) to get a sense for how often it finds or misses the known working path.
In a way LLMs are, after all, trained to LARP people, including fictional characters and their tropes - this was actually exploited for jailbreaking to good effect in the late pre-agentic era (read: some two years ago). C.f. Waluigi effect. Not sure if it still holds for current models, but I can't imagine why it would not.
First, is it reproducible consistently at ~50% of workers? If not, what is the rate.
Second, are there any lessons to be learned here to increase the rate of success by changing models/weights/training?
The news by itself isn't really good news. But it could lead to good news. Maybe.
How would they react if a human were to discover vulnerabilities in cryptosystems?
Although if RSA had a vulnerability I would be very very shocked probably because I still haven't learnt post quantum encryption algorithms enough to really feel like they should be unbreable...
If there is a researcher or someone in space how should I feel about it. Is it as bad as RSA being completely broken open?
I do understand that AI will get better, and a lot actually at very easily verifiable tasks but this one I find it hard to wrap my head around because of my ignorance.
She also notes, as a side remark in a different post: "I would say that my trust in lattice cryptography is pretty much equal to my trust in elliptic curves, and quite a bit higher than my trust in RSA." (https://keymaterial.net/2025/11/27/ml-kem-mythbusting/)
Despite HAWK having survived two rounds of expert human review over a period of two years, Mythos was able to improve the best-known attack on it in just 60 hours of work—effectively cutting its key strength in half.
Mythos’s attack works by finding a specific, previously unexploited symmetry called a nontrivial automorphism in the lattice used by HAWK. Prior work proved that efficiently finding such an automorphism would permit an attack, but did not answer if such an automorphism was accessible in the lattice used by HAWK. The automorphism discovered by Mythos allows a faster enumeration attack that, while still exponential, means that one needs to double the size of HAWK keys to achieve the same level of security.
...and it is kind of incredible to think that they spent $100,000 over 3 days looking for an automorphism. Not the possibility of an automorphism, that was already known. Man.
[1] ... or focusing too hard on the strange use of mathematical language...
I feel like there is a pattern emerging regarding the type of novel discoveries LLMs are good at finding, but it will take some more data points to see if the trend solidifies.
The attack on HAWK is perhaps more interesting - they were able to halve the effective key length. HAWK is a candidate for NIST standardisation. It has been studied academically, but isn't really deployed anywhere (because it hasn't been standardised!)
It is standard in cryptography to analyze ciphers under this kind of attack, which is stronger than normal attacks, because a cipher that resists to a stronger attack will also resist to weaker attacks, so using the strongest possible attack increases the confidence in a cipher.
While using the strongest attack for testing a cipher remains the correct method, chosen-plaintext attacks are no longer realistic today, so even when a cipher appears somewhat vulnerable to such attacks that does not imply that it is vulnerable in normal use.
The reason is that the modes of operation for ciphers where the base cipher can be attacked with chosen plaintexts are obsolete. The most frequently used modes of operation are now modes like the counter mode (e.g. in AES GCM), where it is impossible to perform a chosen plaintext attack (i.e. where you must trick the victim to encrypt a text that you choose, but in counter mode the cipher only encrypts a sequence of numbers chosen by the intended victim, which cannot be influenced by the attacker).
There's a push to turn off the classical modes and rely entirely on PQC for both quantum and classical security. Uh... no, thank you? Why would we want to do that at this point? The classical cipher component isn't hurting anything. Awfully creepy to pushing reliance on the new thing alone.
... especially now that we have LLM-discovered attacks on the new things.
HAWK is a signature algorithm, not encryption.
That makes attempts to push PQC-only modes super suspicious to me. Smells like Dual_EC_DRBG.