Posted by ropbear 14 hours ago
So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illicit means.
Any university using AI detection in their submission pipeline, or lawyers, editorialists, proofreaders that check for AI marks will be sending significant amounts of text like unpublished research, books, potentially internal documents and more, most of which is high quality human written, to dozens of AI companies, blindly trusting they won't train on any of that.
The upside is that this has very low false positive detection rate, but the downsides are many. It only works on longer pieces of text. The system is fragile, and small edits (or rewrites by a local model) can fool the detection. Only the owner of the model is able to re-run inference at this level, so data must be sent to them for evaluation. And sometimes the token output is basically 100% deterministic because the input asks for the straight answer to a fact, or to recite a quote verbatim. That leaves no room for watermarking at all, unless the model is able to lie.
I suspect the signal will be significantly under the noise floor, so it's not detectable if you don't know exactly what to look for, but certainly you can submit more information then the textual contents.
I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!
They're giving you an oracle regardless, which is almost as good. Take LLM output, make some modification, ask the detector if it's LLM output, repeat until you learn what kind of changes you have to make to defeat it.
Or don't even bother learning what to do, just make arbitrary changes until it says it's not, so when the person they're submitting to does the same check it says the same thing.
Alternatively they could also just keep saying "yes" if it's close enough to a version that was close enough.. Although that would enable the attack to allow arbitrary text to be "proven" AI, by slowly morphing close-enough generated material to the desired text. But perhaps this is not a problem they are not concerned with.
To satisfy the letter of the law I expect it's enough to just provide the oracle, without any mitigations.
Reference needed? I think it remains to be proven whether those detectors can be considered deterministic.
A good fingerprint should make use of cryptographic signatures. Without knowing the keys, the fingerprint should be indistinguishable from noise (or just random token selection)
its also kind of laughable that somehow people are trying to prevent the outputs not to be altered. Asif you cannot manually paraphrase anything you can read. So the only solution would be, to make it utterly unreadable (which is not possible, it obviously defeats the purpose of the thing).
Not to mention local models ofcourse :-)
Having proof that content (especially images and video evidence) is unmodified (whether via Photoshop, Paint or a model) is far more valuable then having evidence that an image was manipulated or generated fully by a model (which still leaves other forms of manipulation), I feel the same goes for human authored vs generated text. Free to admit that using models to generate any kind of media whole-cloth is still unappealing to me and I still pay for commissioned artwork or make it with my limited abilities for what that's worth. Do like to (poorly) write my musings too and see UX as something were thoughtful contributors (like the opinionated, sometimes controversial, but certainly talented GNOME Gitlab contributors) can make a major impact.
Code can be beautiful, interesting and serve purpose beyond execution, of course, but for most people, in most cases, it does not in the same way as audiovisual content (not limited to art). Having code just to execute and resolve a problem can have value all in itself, the code being a means to an end whose quality, let us be honest, was barely a concern in most corporations long before LLMs.
Also have rarely (honestly never) before LLMs fully owned all parts of any code base, always relied in part on someone's prior effort in (Flutter/Dart mostly) packages, whereas when writing, drawing, etc. I have far more situations where I make something from scratch and everything there is only there because of my conscious decision. Even simple marketing mockups that, quality wise, any modern model would beat feel different when I was fully in control, where to place what, etc. Objectively worse (at my skill level), probably, but still never the same.
Knowing something was made from scratch by a human has value to me, beyond misinformation prevention. Knowing for a fact that LLMs were used instead of importing a library, using a template, or something similar that leads to expending similar amounts of effort, I don't see that being nearly as valuable. Heck, with all the importing and my experience back then vs now, I am spending more effort actually fully reading any LLM output in my code then I spent back then auditing Flutter/Dart packages. Then again, LLM output fails far more unpredictable then those messy packages that simply got Gradle to take down my system...
Happy to admit, I have been skeptical of watermarking LLM output being feasible for quite some time and having looked into SynthID Text and proposals being researched, I am convinced that it is challenging to impossible beyond the lowest common denominator and less important then proofing human authorship.
It will catch people just copying LLM output into their replies without thought, which is not a negative in my book, especially if it is not discernibly affecting output quality in regular use cases. Anyone who wouldn't copy Wikipedia into their dissertation will, in my opinion, be able to bypass text watermarking as proposed however, I feel we need to be honest there.
Thing is, if that's the case and text watermarking will only ever catch LLM created slop, is that a bigger problem then the misinformation, harm to creators due to authorship questions and accusations, making it harder to use evidence in proceedings, teachers not trusting students even when they did the work themselves, etc.? Signatures for all such cases will be difficult to implement, yes, but I feel are going to be of greater value in the not to distant future and I equally feel are not impossible, not least because idiots will always want to hide their LLM usage, whereas human authorship is something they take pride in and want to proof.
Artists, photographers, journalist, etc. are going to want and need this.
Because a malicious human will gladly copy/paste LLM text and sign it with his "I, a human, definitely wrote this academic paper" key?
being allowed to train on any data that you can legally obtain ought to be a right for anyone.
After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.
I have the opposit viewpoint to the extreme. They shouldn't be allowed to even read that data until they are very clear about what they will or not do with it.
Can they publish it? Can they store it? Can they use the information in it on prediction markets? Etc.
Humans reading texts historically come with little negative consequences, but machines reading and processing texts en masse is more dangerous and should be regulated.
Citation needed. This is sounding tautological.
It's part of why we sign NDAs, and why their duration is measured in years (and that's not even targeting the human retention - just duration after which information ages enough that its disclosure is not likely to negatively impact anyone who cares).
Are you a tool?
Because humans gets rights, tools don't.
Arguing that untrained or partially trained models should have have rights is a different argument to arguing that a trained model should get the same rights as a human.
But LLMs are replacing the original, just in different words.
And what does 'legally obtain' mean in this context? Copyrighted content is usually licensed for specific purposes. So if a license is given from training your LLM, then by all means do! But what if the license is 'for personal use'... ?
Correct, if you violate it too often to count, you have to pay around less than ~2.5ct per violation.
So the lesson here is: Create a company to do torrenting professionally, and resell its values for higher prices. Then get sued and pay a dime on the dollar you made.
edit: Actually it's 2.5ct per violation.
Why should we hand over even MORE power to the owner class?
In a fantasy world this could be possible yes.
It's a good idea for many human endeavors to be able to identify AI writing. Communication, after all, is our main way of building the social fabric.
However - and crucially - good writing is still beyond the frontier of any model I've seen so far.
Watermarks for the things that truly matter may not be important at all.
Finally, as X commentators have shown, simply removing punctuation or changing a word here or adding an adverb there manually will screw up the whole process enormously.
The best will be the clever folks who retroactively apply the model distribution to fraud or other crimes to try to implicate the companies via watermark.
Gotta feel for their product, policy and legal team.
Then write your own damn text if you care about the exact wording so much
If you want precision and clarity of your writing, then you need to hand write it. Just like when you are optimising, its common to drop to a lower level language because the compiler doesn't express what you want. Sure its hard, but you know, thats kinda the point.
Even if you don't want that, the LLM is an average of the style it was trained to give out. Which is a homogenisation of the language to create a vague padding medium between a few generalised facts. (because a. it makes it less jarring when stuff is wrong, because its smeared over a higher amount of text and b. it looks more 'professional' because American business English is all guff and no meat)
Also yes, two isolated phrases may have subtly different meaning, frankly, the nuance is missed on most people. If you look at the interactions on here, at least 25% of the arguments are caused by people angrily reacting to the things _they_ thought the other person was saying, rather than what the actual person was saying.
So no its not a perversion, the LLM is, if you're gonna be picky about things.
exactly. in the same way that printed books affected word choice, so did the radio.
Watermarking per model is just the start. The method is cheap enough to distinguish individual users.
If they are adding so little value as to be as transparent as a pen and paper then why use one at all? Transcription doesn't need an LLM so that's not what you're taking about I assume.
Then use pen and paper. It is the same, you say, right?
The problem is, that LLMs worked very well for me to improve my writing. Especially as I'm not a native speaker, it was a great way to improve the legibility of my work.
I want a tool that helps me improve my writing. A tool I can learn from. Not a tool that switches out "bananas" to "airplanes."
I'm was using Claude Opus and now Fabel extensively for editing my texts and I find the recent updates abysmal. Not sure if it's due to the Text Watermark.
Before Claude was great in sharpening the meaning in my writing, it's now close to unusable.
The only difference here is that Anthropic is actively trying to make the watermark undetectable.
When writing in English though, I use it more like a dictionary. If you want to write past a certain level, an LLM works better as a metaphor and idiom search engine.
I also like to ask it to generate 20 ways to say the same thing. It’s a great way to simplify or smoothen sentences without losing your voice.
I don't expect the LLM to read my mind. The unit of work is too small for intent to matter, and I'll just steer the next recommendations in a direction as needed.
Most of the suggestions are crap, but they can contain the seeds of a good sentence.
I know that most people don't care, but my online presence is a search query for interesting people, so I care about what I put into it.
Nitpicking here in a way that I would usually avoid, but it is relevant to the conversation being had and it seems like you might appreciate the information... "Readability" would be the more correct word to use here instead of "legibility".
Legibility is close enough for me to know what you mean based on the context, but it really applies to the visual presentation and how easy something is to read at a symbolic level (whether someone's handwriting or font choice is good or bad impacts legibility, whether someone uses good grammar or not impacts readability).
This comparison is frankly absurd.
Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo random. The output it generates is one of the possible outputs it would have generated before, just now it's deterministic and will generate the same thing every time.
> To illustrate, in the special case that GPT had a bunch of possible tokens that it judged equally probable, you could simply choose whichever token maximized g [a cryptographic function]. The choice would look uniformly random to someone who didn’t know the key, but someone who did know the key could later sum g over all n-grams and see that it was anomalously large. The general case, where the token probabilities can all be different, is a little more technical, but the basic idea is similar.
(1) The behavior that is approximately what you describe is not "fundamental" (though it may not be something you can disable on some hosted providers), it is an option that is not fundamental (and with runtimes where you have full control can be either disabled or tuned in a large number of manners), and
(2) The actual behavior that is approximately what you describe already usually involves use of PRNG (with a user or harness supplied seed), not a true RNG; the change to do watermarking isn't going from RNG to PRNG, it involves adding an additional set of constraints on token generation on top of the existing ones, which inherently compromises quality.
In case of LLMs, you can look at it from high and low level.
At low level - if you could do with less randomness, you can always lower temperature. You usually keep it (or for SOTA providers' chat UI, they keep it) at a level where it's about right level - high enough to allow for more creative leaps and interpretations, low enough that it doesn't go off into crazy land after the third paragraph.
At high level - creativity is driven by randomness. If you had an author (fiction or nonfiction) you like for their both broad and deep range of insightful thoughts, would you be happy if they suddenly developed an acute porn obsession and uncontrollably added lewd subtext to every other sentence? Still creative, still deep, but now with that one strong attractor that biases their every thought in a single direction? Would you trust/enjoy their output as much as you did before?
That, slightly exaggerating to make it more obvious, is what "loss of quality" means here.
2) Claude’s PRNG having a P is immaterial
LLMs are likely to get stuck even with sampling if asked to generate tokens on their own long enough, though sampling does tend to stretch out the time before that happens (as do other techniques that don't involve sampling, like applying repetition penalties directly to token logits). But LLMs generally aren't left to infinitely extend their own output, and the length response typically needed in the use case is much shorter than the would result in collapse given the kinds of inputs expected in that use case, the existence of the theoretical eventuality may not really matter.
Related: if you don't have a limit on sampling (top-K or top-P), eventually you'll hit one of the really unlikely tokens by chance and then the model will switch to Japanese because the most likely completion after a random Japanese character in the middle of an English sentence is more Japanese writing, not a reversal back to English.
I don’t see how this follows? Tokens are chosen randomly. If you choose tokens with a different RNG in the same distribution, you’re still getting equally good or bad tokens.
Writing has rhythm, or at least it's supposed to, and synonym swapping compromises it.
Never mind metaphors and similes, which are even more tightly constrained.
LLM writing is still a long way from good. Sometimes you get lucky with the odd line, but there's a difference in quality between influencer slop, genre fiction, and literary fiction and/or best-in-class journalism.
LLMs are still somewhere between the first two, and nowhere close to approaching the third.
We already know that a non-zero temperature improves quality though with current models (particularly with creative writing). The assumption that always picking the 'best' token results in the 'best' output is not the current reality.
And if you are already intentionally putting in randomness, I can imagine that it would be possible to seed the randomness in a way that is detectable but results in the same quality.
This is obviously not true for queries where temp = 0, but at temp = 0 then it becomes easier to identify anyway. I assume this technique implies some level of temperature.
On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense.
Even good human writers would probably struggle with this constraint. It would be like someone interrupting your writing to tell you the next word MUST be such-and-such, and then you have to try and make it work as best you can first try, without going back to edit. The result would probably be a little clunky. (Maybe it’s impressive LLMs write as well as they do.)
You're mixing up two claims here, and only one of these is kind of true. Yes LLMs do internally plan ahead in a way that is emergent rather than strictly part of their architecture, so that part of your claim is true. The way you word it by saying they are "coalescing the probabilities of a range of tokens at a time" is poetic sounding jibberish though. What's actually happening is one distribution output for the next token computed from a hidden state that implicitly encodes where the text headed.
Your claim that if an LLM does happen to pick a token "th" instead of "tw", then the LLM isn't stuck with that decision is entirely false for autoregressive LLMs which is what all of the frontier models are. Whatever an LLM picks as its output token is final, it has no ability to undo that token selection and it must continue on the basis of that choice. It can't go back on that decision and revise the output.
If you're interested in this, Anthropic has a summary of a very technical paper on this topic that mostly deals with this issue with respect to poetry:
https://www.anthropic.com/research/natural-language-autoenco...
So we train a second copy of Claude to work backwards—reconstruct the original activation from the text explanation. We consider an explanation to be good if it leads to an accurate reconstruction. We then train Claude to produce better explanations according to this definition using standard AI training techniques.
The choice is between "this reconstruction sucks" and "no reconstruction", and we're only now beginning to learn how to make those reconstructions suck less.
Mathematically, a long chain of conditional probabilities is equivalent to a single probability over the whole range. But computationally, for that to work out, the computation for the first probability needs to somehow consider all the downstream probabilities depending on it, which obviously isn't how autoregressive language models work. They can pack in as much downstream computation as their neural architecture allows for, which is quite a lot.
Suppose in some context you have three equally plausible conpletions after "Be": "tween a rock and a hard place", "twixed he stood there" and "lieve he can fly". To model this probability distribution of the whole sentence, the next token "tw" needs to appear at 2/3 probability and "lie" at 1/3. After "tw" would be a 1/2 chance of "ix" and a 1/2 chance of "een"; after "lie" would be a 100% chance of "ve " and in any case the rest of the sentence after that would be 100%.
The model needs to somehow "think ahead" to know those are the possible completions. For example if "lieve he can swim like a dolphin" was another equally plausible completion, that first token would need to be 50/50 instead of 67/33. So the computation of the first token somehow needs to encode the fact that the guy thinks he can fly but not swim, even though it doesn't become relevant in the output until several tokens later.
In practice this probably happens to some degree but definitely doesn't happen perfectly. To perfectly model the first token's probability distribution, it would have to include knowledge of the entire distribution of all possible outputs, which is just not happening. So it approximates. Surprisingly, the approximation is good enough to produce language.
You can see this breaking down in the seahorse emoji incident from last year. When you ask the model if there's a seahorse emoji, it first completes "Yes," as if a few tokens later it's about to produce a seahorse emoji. But when it actually gets to the token that would produce a seahorse emoji, it can't because there isn't one. But it's already outputted "Yes, the seahorse emoji is" and can't just go back and change that to "No, there's no seahorse emoji." Some models would try a few times and then say there isn't one or a system error seems to be making them unable to produce one, other models (including then-current ChatGPT) would loop forever with ensuing hilarity.
But also, no one really knows as they're closed.
https://chainofbranches.com/conversations/2/branches/20/
I’m not convinced it’s possible. A good nights sleep and a notepad in a quiet room still feels like the state of the art toolchain for writers.
Presumably you could use the same reasoning trace, run multiple generations, and get different outputs (if the temperature is >0).
But now I’m interested in playing more with Cowork or Claude Code/Codex for prose writing to see if the set of tools there affects outputs at all. I guess you might need a more custom “writing” harness.
Reasoning tokens with tool calling tell the model to loop on a one phase of a question and call a tool to indications completion when done.
Related, but not the same thing.
That's what LLMs in reasoning mode do, too, to the text they present to you.
I feel like the existence of good writing is also not impossible but not very likely, and so of course LLM can only write mediocrity, even when taught only on great writing.
Here’s an example: I had asked Claude for some music recommendations in a certain style. Part of its output was:
—
*Long journey tracks*
Clinic — “The Return of Evil Bill”
Guided by Voices — not really, wrong band
Silver Apples — “Oscillations”. Proto-everything, deeply repetitive, hypnotic.
—
So at some point there, the next token produced was “Guided” or “Guide” or whatever, and then because it can’t go back, it had to correct itself after the fact.
Reasoning/CoT have helped a lot, but I feel like small versions of this still happen all the time.
Human writing is like 90% editing.
But that would be a fun writing exercise, I think. Thoroughly in the oulipo wheelhouse.
Maybe generate a Markov chain table over all of Project Gutenberg and then say every 10th word is whatever the Markov Chain thinks it should be at that point?
Or every Nth word has a P% possibility to be constrained by the chain? Optionally with the possibility building for each skipped word to guarantee it happens at some point. Bonus with this approach is that the human can't game the words leading up to the constraint because you don't know when it will happen.
Well, I suppose it's nearly the opposite of that experience, upon further review. But for some reason, that's where my head jumped.
You be a human who's brain shifted into LLM mode (chainneling Markov?).
Or perhaps you're an LLM impersonating humanity.
I often wonder how much LLMs are just mirroring our own brain's patterns.
But what about the general idea that they can watermark results to tell where they came from. The next step is tracking down which user got a result. I hate both of these things. Must everything we do be tracked? Next altering wikipedia results so they can tell who looked at the page or something?
I'd like "the best answer" from an llm and don't want to be tracked, but this isn't for me, it is for them. I understand llm results are already using a varying statistical input so they aren't always the same. But I really hate watermarking and likely tracking too.
They are also usually worse (which is often better!) because they are usually lazy and don’t want to spend effort they do not have too, to accomplish their goals.
Their goals are often complex and nuanced.
None of this is true of LLMs.
Watermarking changes the probability calculations for reasons other than quality. It can't not compromise quality. It literally leads the LLM to occasionally chose different tokens just for watermarking purposes.
Also, as long as the same sampling strategy is used during training as the one used during inference, then the LLM will actually do much better with the biased sampling strategy than it would with a fair one - because that is what it was trained to optimize.
So what? By definition with this system the LLM will chose tokens it otherwise would not, purely for watermarking reasons. Yes this token may have had a decent likelihood of being chosen anyway, but it wouldn't have been chosen and now it was for reasons nothing to do with output quality.
I'm not sure what your last paragraph is trying to say. The blue/green list system changes what output the LLM would otherwise produce. You can't train it to produce watermarked output with this system. If you tried to, there would be no delta between trained output and watermarked output for you to be able to detect.
We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user.
Fundamentally the article is railing against --temp > 0.0. He doesn't know what he's talking about.
You can't as a user tell by how much the quality of the output was degraded. True.
>We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user.
I'm not saying it wasn't random and now it is. I know how these things work. I said that the quality of the system is in the quality of the probabilities. That quality is being degraded.
Even so, I don't think it will stop here. Once this is in place, the next step is to put more and more identification into the AI generated content; might as well pack it in, it's not that bad, and if it is they won't admit it. There's no way for anyone to check. And your argument will still be technically correct but missing the point.
In fact we know it's not that good because we can often tell Claude's writing apart from human writing.
Could it have been equal or better with slight variations in wording?
The slipper slop argument is too lazy to address directly. Argue A is bad because A, not because A might become B and you’ve got good arguments against B.
I think there is a possible weakness in the context of the watermarker but that is not your claim iiuc.
I'm skeptical that anybody generating LLM text is really all that concerned about optimal word choice. Or even particularly good prose. But let's pretend that person exists.
If that person tried, say, an open model and that same model with watermarking applied, I'd be eager to hear their thoughts on the prose quality. Especially if they built an experiment harness and rated a few hundred blinded examples and found a measurable difference.
But getting this upset in advance of any demonstrated problem? It really seems to me like the point isn't the point
Yes, but that's neither surprising nor a reason to dismiss the anger. People get angry about DRM schemes in video games, even if the slowdown these cause is practically imperceptible. They're angry -- and Gruber acknowledges that factor too -- because a stranger manipulates what they regard as their own domain, without consent by or benefit to the owner.
It might be another instance of consequentialism vs. honor ethics. Many consequentialists don't seem to understand that something that doesn't have demonstrable consequences can still have moral implications.
Yes but thats a thing that degrades something in a catastrophic way, as in I can use the thing one day, and not the next.
A different randomisation system on something that is a text generator which is designed to be unperceptable sounds like the people who are annoyed at FLAC vs MP3[1]
Done right you won't know the difference, done badly and you will.
[1] ex audio engineer, try me.
Any potential "slowdown" doesn't even come close to making the list of top reasons people get upset about DRM.
> because a stranger manipulates what they regard as their own domain, without consent by or benefit to the owner.
It's LLM output! It's not your domain, it's the LLM owner's!
that's been his thing since it was just a blog about apple product speculation and update. It's always been tedious.
The easy thing to do here would be to have 1000 questions, randomly assigning one half to an LLM with a watermark, and the other half without. Then show people pairs and say, "Which one seems watermarked?" (Or, "Which text seems more natural" or "Which is a better answer" or something like that.) If they come out equal, the watermark really is indiscernible, at least to most people.
I think in this case it doesn't help that there are multiple watermarking schemes, and the easiest for people to understand is the red/green scheme by Kirchenbauer et al. (https://arxiv.org/pdf/2301.10226), which does technically distort the logits (but I'd argue only in cases where you wouldn't notice it anyway).
I wasn't aware of this gumbel softmax scheme, it seems you're referring to https://simons.berkeley.edu/talks/scott-aaronson-ut-austin-o... ? That's really clever as it doesn't even distort the logits, basically cryptographically indistinguishable from a "real" random sample unless you have the key.
The actual scheme Claude uses seems to be neither of those two though, they say it is SynthId-text which seems to be tournament sampling based.
Cognitive surrender.
How does that follow? AI-generated text is already not a perfect emulation of human writing. There's lots of room to affect it laterally without changing the level of quality.
As I understand it, LLMs with temperature >0 can select from many possible outputs. All they're doing is limiting the possible outputs to ones that contain this pattern. I don't see any reason why the quality of that subset should be lower than average. The very best outputs will likely be eliminated, but so will the very worst.
If you get your random numbers from a cryptographic PRNG, then to notice the difference between that and 'real' random numbers even in theory, means you need to break the cryptography. In practice, your gut feeling about how good some text is won't break modern cryptography.
The "problem" is that seeing the watermark doesn't mean that the person claiming to be the author didn't make extensive changes to the output of the LLM, or that the LLM wasn't simply the final editor of something that the author had put a lot of work into.
> Cognitive surrender.
I don't know what this means. It's just drama. Don't let the LLM write for you and this is not a worry. I'm not worried about the poetry of LLM output being subtly adulterated.
Prove it, then? It's not a claim that GumbelSoft paper makes: "Regarding generation quality (perplexity), GumbelSoft shows relatively low perplexity"
Indeed.
It's frankly bizarre to see the assumption to the contrary being made by someone who's been passionately blogging by hand for years, who also happens to be responsible for the notoriously vague, humanistic, DWIMmy Markdown standard.
On deeper tech stuff, like this utterly nonsensical misunderstanding of watermarks… yeah, classic case of a guy who is smart, and has lost the ability to realize when they’re not knowledgeable in a domain.
> At each decision point, they’re a little more likely to pick a word from the green list than the red list.
Wrong. There is no global red and green list. It's dependent on context and balances out on average. It won't change the result when one token is predicted overwhelmingly likely.
Well, not that weird actually. He just has a hard-on against anything that comes from the EU since Apple got in trouble. If the EU said tomorrow that they want peace in the world he’d be in Fox News the next day calling for an invasion. As a former reader of Daring Fireball, it’s just sad to see.
I dunno, I guess that's what you should expect from Gruber but these EU-bashing articles lowered the enjoyment I got from his blog underneath the bar for me.
Further he later compares Gemini to Anthropic models, saying the latter "writes better", emptily ascribing this to the synthid stuff. I think he heard that Anthropic currently has superior models, but it certainly isn't because they "write better", and if anything Opus 5 now is virtually unintelligible, before the fingerprinting.
The fingerprinting stuff sounds weird. If the EU wants it, it should be limited to the EU, and Anthropic is fully capable of doing that but clearly saw value in recognizing their own output. Is it going to destroy the quality of the output? We'll have to see, and this anti-EU piece, predicated on utter ignorance of the field, is not convincing.
I think they need to get their shit-together and realize this is a death warrant for the tech ( in my opinion ).