Posted by erwald 3 days ago
This is the best articulation I've seen of why simply reviewing and copy-editing does not provide remotely the same value as writing from scratch. I spent a considerable amount of time over the past two weeks reviewing and improving a work document that was the output of an LLM. Given the number of people involved and the final level of effort, I'm firmly convinced that writing it manually would have been faster and resulted in a higher quality product. Getting the wording right matters.
Why would you think that asking someone else (that is, another human) to write something (and then reviewing it) is the same thing as writing it yourself?
You may trust the other writer's opinions and knowledge, but it will not have the same tone, structure, word choice, understanding, or narrative flow as it would if you were to write it yourself.
And when it's an LLM, you should not trust it's "opinions" and "knowledge", because it does not have either of those things. The appearance of those things is just that, an appearance.
How do you recognize someone as having true opinions and knowledge from someone having just appearance of them?
And there are countless other jobs out there where people are the “voice” of another party. From those who manage social media presence, to PR firms, to copyrighters, all who put statements out on others behalf.
There was already an industry of professionals whose job it was to have others write things on our behalf, and that existed long before LLMs were a thing. What LLMs did was make that that service available much more cheaply.
And no LLMs are not producing ‘the same thing’ as a speechwriter. If you are unable to tell the difference I’d suggest doing some a lot more reading of human books. LLMs produce stultifying pablum without coherence or style.
Nope. The exact opposite in most cases. Eg Jon Favreau (Obamas speechwriter) has said:
“As a speechwriter, your ego has to take a backseat. The goal is to make the speaker sound like the best version of themselves, rather than to showcase your own cleverness."
> And no LLMs are not producing ‘the same thing’ as a speechwriter.
I didn’t say it was the same. I said it was an existing industry that AI has stepped into and AI is taking the low end of market.
It’s just like how AI has captured the low-end of many tech roles too.
> If you are unable to tell the difference I’d suggest doing some a lot more reading of human books.
“Some a lot more”? Very ironically timed editing error there. You can bet an LLM wouldn’t have made that mistake ;)
AI has replaced neither full tech roles nor speechwriters.
I think that’s a good example of the kind of trivial error humans make all the time when not rereading/editing. When humans make mistakes, it’s usually that kind of mistake, which IMO doesn’t really matter in an internet comment.
Unfortunately LLMs make gross errors of style and content and often just don’t make any sense in long form text. That’s a very different category of error.
That would explain why he might say that if directly asked. But not why he volunteers that information nor talks about his research into Obamas speaking style before writing his first speech on Obamas behalf. Nor any of the other detail the he, and other speechwriters discuss when talking about their job.
You accused me of not reading enough before, and yet here you are making bold claims about a profession you’ve clearly not read enough about.
> AI has replaced neither full tech roles nor speechwriters
As I said in my previous post: i claimed AI had replaced all people. I said it’s captured the low end of the market.
> I think that’s a good example of the kind of trivial error humans make all the time when not rereading/editing. When humans make mistakes, it’s usually that kind of mistake, which IMO doesn’t really matter in an internet comment.
If you’re referring to your error, then it was still mistake regardless of the nature of it. And yes it does matter. It matters because if you want to talk about professional-quality writing, it’s the kind of mistake LLMs wouldn’t make.
> Unfortunately LLMs make gross errors of style and content and often just don’t make any sense in long form text. That’s a very different category of error.
Your comment technically didn’t make any sense. I was able to extract the intended meaning, but that wasn’t because your comment was well written.
And if we are going back to the speechwriting example, such prose isn’t read to an audience without the speaker reviewing and iterating with the speechwriter. And neither should LLM output be submitted without review.
Your argument here is effectively saying “AI isn’t as good as any humans because I expect it to perform better than the top tier of professionals.”
Clearly that’s a biased benchmark.
Whereas what I’m saying is “AI isn’t as good as top level professionals but it already offers a good-enough alternative for the entry level requirements”. Which is a completely different yardstick.
And all of this arose from your original comment that AI mimics voices so isn’t any use while ignoring the evidence provided that professional writers do exactly the same in many fields. Again, you’re deliberately skewing the facts to suit a position you’ve already pre-determined.
Now I’m not going to argue that AI is a net benefit for society nor that AI is an ethical technology. Frankly, I prefer life pre-LLMs too. But that’s a personal opinion separate from the facts of the conversation.
Obama, former president of Harvard Law Review, is far from inarticulate and unpolished. The truth you are ignoring is that speechwriters free the speaker from long hours of speechwriting, analogous to a chauffeur enabling them to work while commuting (chauffeurs are not principally used by incompetent drivers).
Every politician - even Trump - reviews their speeches and edits to better fit their desired goals. Speechwriters are principally labor-savers.
The whole catch is that they can often be regenerated. But LLMs (on their own, in their parametric memory - which is the result of training) don't have any conception of whether what they've generated is a real reproduction of some training material or whether they've invented something false that seemed probable based on their encoded stats. When the probability produces something contrary to what was in the training material, you get hallucinations.
They're very, very good predictive text models and can be very, very powerful when hooked up to other tools or outside databases. But its fundamentally lossy technology and all the books having been fed in doesn't guarantee all of the knowledge from those books can be spat back out.
Every performance- singing a song, playing an instrument, performing a stand-up routine, giving a speech, performing a theatrical role are all things that are best done from memory but require practice.
There are pneumonic tricks you can use- I've seen some people do it for tricks like memorizing the order of a deck of cards- but it's less useful for long term recital because it helps with order but not comprehension or fast indexing.
I don't really understand this side of the debate other than as a gotcha tbh.
If LLM use atrophies your brain and skills that's bad. If it has a repulsive writing style that's bad.
I'm not sure what the debate about whether an AI is a statistical parrot unlike humans accomplishes. Is relying 100% on a bad human speechwriter somehow better?
The primary complaint seems to be that LLMs are held to a higher standard than humans, though I don't particularly buy that line of reasoning.
But in professional settings, a lot more of the informativeness is about the author, and a lot more of the persuasiveness is I'm worth your time and money. So, if the author is an LLM, and obviously so, what exactly are you informing your audience of (about yourself), and what are you persuading them to do (with your article).
I think we now know.
LLMs are great to make drafts if you give them the source materials. They're great at validation if you give them the tools. They are great at layouting if you give them linters.
Encode architecture and decisions in your workflow, then proofread what your agents have been working on. Not the other way around.
Better validation and testing means more work will transform from exhaustive decision work to automateable gruntwork.
Meanwhile, me, as an English non-native speaker, ended up discussing two sentences I want send to HR for ten minutes while applying to a job.
I do believe there's generally a bias to accept something that's already written. The much bigger reason though is why you let somebody else write it to begin with.
It might just be that not thinking carefully about every sentence/wording was the exact thing that made you use AI to begin with.
> I spent a considerable amount of time over the past two weeks reviewing and improving a work document that was the output of an LLM.
I had the same experience with texts where I have a very detailed expectation of the desired result. This is just a general limitation. For code, there's the saying "the precise description of the solution is already the code". Describing X is a simplification of X, oftentimes it's fine guess the gaps. But when it's not, it didn't help to describe X, you have to manifest X itself.
But in your own mind, your thoughts are still incomplete. The act of writing them down (or, I imagine, in an oral tradition somehow committing to a specific narration) is very important. IMO, when you delegate that to an LLM, you are not really thinking. It's the same as if you explained your ideas to someone (eg in an interview), and then they ghost wrote the work for you.
And then if you want to tell a message with a different tone, again, you must consider differently.
The considering can be strengthened with exercise.
I can't imagine worrying about signaling as I write: what a huge distraction.
The problem is: this type of thinking and writing takes a lot more time and attention.
How much consideration should you put into a message? It's a personal answer, but also one that can be constrained by time.
I was talking about the step before that: what are your ideas?
How do you circle an idea in your own mind: your train of thought, in order to express an idea?
Vocabulary to think, and then if we need to share, we use careful to pick words with the reader in mind.
But just in your own head they lack clarity. I think this can be deceptive to people, it’s a blind spot. How often have you sat down to talk over some disagreement with someone only to find their own thoughts are a jumble of disjointed ideas that don’t make sense?
Some Jazz musician was asked to define Jazz.
He couldn't really, his answer was something like: 'I don't know. But I'll know when I hear it'.
To add something to the discussion directly: thinking requires vocabulary. Vocabulary is the currency of thought.
You can usually express an idea with a few thoughts, or many. The audience, and amount of details chosen should always be kept in mind. Writing helps you to remember vocabulary and word choice when expressing ideas.
Not that it's necessarily better but it's the kind of thing an editor might pick up and so why would you reject the advice if it's a machine giving it rather than a human?
See also this existing comment: https://news.ycombinator.com/item?id=49768564, which I completely agree with and which is complementary to the above root comment.
What do you see as the difference?
If you want to discuss something else just let me know.
Going from foreign to native is easier than native to foreign. The latter requires completely different brain paths and a lot more understanding of the language to actually get to something correct.
Don’t use AI to write things that you are producing for someone else to consume.
I'm not saying you should never do this: our time is valuable, and we shouldn't spend it on things that are not genuinely worthwhile to us if we can help it. But we're still losing something by having an LLM write for us, even if the intended audience is just ourselves.
I do not personally even think AI is useful for editorial purposes; I would rather read and re-read my own text and edit it down (which, despite my overlong comments here, I actually do), than have someone else do it. The reason is simple: editing is my search for the clearest way to express my thoughts.
I tend to turn grammar checkers off, but leave spelling checkers on, because they catch typos and a handful of my spelling gremlins like "liaise".
They can be broken and jumbled, or terse AF, or combine to be Pulitzer-quality prose. Either way: I want to consume their own unique human expression of a concept.
There's value in raw human expression, including in the missteps.
If I instead want the regurgitated waggyings of a bot, then: I know how to get that on my own.
We've all got web browsers and pocket supercomputers. We've all (well, most of us) been alive and aware of LLMs since their recent rise from infancy.
It's a no-brainer for us to paste some paragraphs into our favorite chatbot and get a summary or an artificial expansion or whatever else we wish to have. If that's what our goal is, then we can do that on a whim -- and we can still retain the original expression.
But doing it on our behalf is deleterious, unsettling, and unhelpful. It has negative value to the beholder.
There's a (probably subconscious) reason for you wanting that (and I just posted this elsewhere on a different story, but it bears repeating): https://news.ycombinator.com/item?id=49767867
Elsewhere it’s painful and irksome when you didn’t ask for it
Perhaps English is not your first language? To people who are well-read, LLM-ism grate on their nerves like the high-pressure sales pitch you hear on 3am shopping channels.
The exercise of writing is an important step of actually understanding your own thoughts properly. Readers are reading to get a sense of your own experience and ideas on the topic, if it turns out you didn't actually write the final thing yourself they'll always feel a sense – in whatever way – of being conned by the author. Intentional or not.
Technical documentation, where what the reader wants to come away with is the clear facts on what something does and how to use it/whatever, is a different matter. But even then editing is key – overlong text, unnecessary information, these will again fight against what the reader is there to get.
With that said, if you do want to do it, a disclaimer is at least upfront and may well show there's no intentional attempt to deceive.
I don’t agree with this one. Writing technical documentation can be a slog, but quite often you will discover very annoying design decisions in the process. Like you’ll write a tutorial for a CLI and realize that actually some flag makes no sense, or some default should be changed.
This also goes for dogfooding. If you’re using AI to write and run the tools, try to use them yourself and see how nice they are. AI will happily run whatever jank it’s working with.
Docstrings perhaps, but even then I find myself editing heavily to avoid cruft sneaking in.
The other difference is that, if I get AI to write something for me, I expect something AI-written. If I am writing something for you, you expect something that I wrote, not something that AI wrote.
So the intellectual responsibility is diluted to a substantial degree: The text is not the opinion of anyone who can be expected to honestly defend it. It's bullshit in the technical sense.
The same is true if the thing they both want is a stale cupcake bought from a gas station. But this still doesn't mean that person B wants to be presented with person A's stale thing, because the overhead of the social transaction isn't worth the object. Which is to say, it makes a terrible gift.
TLDR: Communication from a human is not only the material being communicated, it's also a signal that the human understands the message they are sending you. When you pass that message via an intermediary (regardless of whether the intermediary is a human or an LLM), the receiver is never sure that what they received is what you intended.
It's doubly worse when you review it, notice a few additions and think "Great - that makes me sound smart and knowledgeable about this topic", because the receiver now has an inaccurate view about your skill.
At any rate, using an LLM to construct a message for someone else to read is basically sending the wrong signals about your skill levels.
(This probabyl explains why so many people are so ready to use an LLM to write the material - in their minds, it makes them sound smarter than they actually are, but as we've seen from people on this forum vigorously defending the practice with "pangram is wrong" and similar nonsense, they themselves are not skilled enough to recognise slop)
100% right. The fundamental point here is:
- it's OK to be wrong
- it's OK to be wrong and not know
- it's a little less OK to be wrong when you should have known
- it's much less OK to be wrong when you haven't put any effort into being right
- it's far less OK to be wrong when you are pretending to know either way
Relying on AI writing risks pushing you further down the path to worse kinds of wrongness.
Yes, in theory, the quality can be the same or better than human output. In practice, the tool creates two problems: it's too easy to produce walls of text for no compelling reason, placing asymmetric burden on the recipients; and it's too tempting to click "send" without reading because you're "90% sure" the email / doc is correct, wasting everyone's time multiple times every week when it isn't.
However, the blog post explains that this is the wrong way to think about writing. The very first point made is that writing makes you understand your subject better and develop better ideas.
Characterizing the argument as 'thinking oneself to be Shakespeare' is a silly strawman and helps nobody.
One of the most egregious negation issues I run into a lot is when I (or someone) makes a statement of the form: "not X" or "X is thus not true", and the AI then proceeds to interpret or summarize this as 'whatever is the opposite of X is the case'". This will cause it to go down a useless path investigating or disputing the opposite of X, which generally has no relevance or bearing on anything.
It also often very harmfully will replace your carefully chosen words with weirdly specific academic operationalizations or formalisms, then again waste huge amounts of text refuting / showing "problems" that result from that formalism, all of which again have no bearing or relevance on the original statement. An example would be you saying something like "intelligence, generally, must surely explain some of the differences in X", and then it will go "actually IQ does not correlate with X", unless you specifically tell it not to conflate psychometric IQ with intelligence generally.
Sometimes this is helpful, but the more specific / technical the domain, the more often you specifically have to prevent it from going down stupid paths that should be obvious given the expert context and wording, because it can seem almost hungry to try to catch you in some kind of insipid 'gotcha'. Much of these issues often clearly arise immediately from the first-pass "reword what the user said" part, given the reasoning traces.
If we don't want to be writers, then we have to be editors. And editing is an entirely different job and it's not an easy one. In many ways it's harder.
Especially when LLMs love writing novels when all we need is a short story or less.
The LLM will always give you a full rewrite — don't use it. It always does too much, and persuading it to tone it down is a constant battle.
Writing code is thinking, AI code is often vague and wrong in hard-to-notice ways, and the (human) reader of code is the one that pays the cost for this later.
The cost/benefit analysis may still work out differently for code though...
My reason: code can be checked objectively. I can run it and confirm it works. I don't get attached to it. I don't feel pride in it (even when I write it by hand). Code just is. It's lifeless, inert, and entirely replaceable.
How do I do the equivalent for prose? How can I tell if my words "work"? Do they clearly convey my ideas to the intended audience? There's an element of subjectivity here forces me to identify personally with the prose.
Code has no such personality. I don't tie my identity or ego to code the same way I would an essay.
Running the code only confirms that it works with the precise input, in the precise environment, under the precise circumstances you run it under. It doesn’t ensure that the code is correct. Thinking through the code, on the other hand, lets you consider all possible cases. It’s the difference between experiment and (mathematical) proof.
For an objective correctness proof, using a formal language is indispensable.
In practice, for code, testing code is the way to go. Formal proofs work, too, but the barrier to entry is high and it’s overkill for most business applications. LLMs can be very good at writing tests, if you read the test thoroughly and analyze them.
Hard for me to imagine. You feel no pride in using a tool to accomplish a goal?
> Code has no such personality. I don't tie my identity or ego to code the same way I would an essay
Code certainly does have a personality. When working with teams for a while you can absolutely get a sense for which person wrote what code in a codebase, just by subtle little tells.
You may not tie your identity or ego to it, bully for you, but for me I take a lot of pride in writing clear and maintainable code that contributes to big projects in meaningful ways.
Maybe the problem with software is there's too many people who treat writing code as a mere means to an end, instead of a very important part of the process.
Code is just sitting in a git repo somewhere, not necessarily running. That's a big distinction for me. Consider that code volume has increased 14x on github this year, but we see nowhere near that increase in the actual usable software. Code is cheap and getting cheaper. Running software ain't.
Another way to put it: I'm only interested in code so far as the value it provides. That value, not the code itself, is the source of pride. If I can provide similar value without any code at all, I'd gladly do so.
The danger for me in LLM code is the same as in writing, it's just that I'm not typically writing at the same scale as when I'm building something. The final piece when I'm writing is usually a message or a 1,000 word article at most. So I'm naturally going to analyze it quite intensely, because I can afford to. And I don't really use LLMs for this at all. I use them for things around writing (research, interrogating ideas, situationally specific stuff, mapping, visuals, publishing, etc.) And the code equivalent to an essay or message would probably be something like a single script, or the sort of thing I'd write as example code when I'm teaching. In those settings, again, LLMs can be helpful, but I'm still going to be really opinionated at a highly detailed resolution.
But a codebase is more comparable to a novel than an essay. Or more directly, the writing in a codebase is usually the documentation, which grows commensurately with the codebase. And the real LLM risk here is the drift that can happen over the course of many epics or "chapters" as the LLM writes "code that works but is imprecise and probably shouldn't work this way" or introduces weird new terminology that neither of us can precisely define. Worse, this usually becomes obvious down the line, and I have to parse through the verbose constructed world the agent has created to trace the issue back. That's a big cognitive tax, because I'm holding these weird parallel worlds of "How did the LLM's alien brain get here within the bounds of the contracts" and "What do I really want this to look like".
So I think it's fundamentally the same phenomenon, and we're all developing our skills around working with it in real time.
All of the arguments here apply to writing code; Why would the quality of writing differe when writing prose vs code?
I cannot read LLM slop prose without getting mentally fatigued trying to figure out what it is saying, and I've discovered that the quality of the code it produces has the same effect on me.
LLM writing takes your prompt and adds stuff you didn't write, burying your meaning and making it harder for the reader to get your message.
Way more effective to just publish the prompt.
Of course, it's great if you're just trying to fill space or satisfy demands of a bullshit job.
I compared at final edited output less boilerplate vs sum(prompts).
If I had to guess I'd say far more people are better at editing than they are at writing de novo. My approach probably limits the creativity of the output but I'm not writing a novel.
Only ever use AI to make yourself think harder, and more.
I agree that being lazy about writing and simply using a short prompt or list is detrimental if you are replacing your own output but you don't have to do that and you don't have to accept any of the output either. The best chats I've had usually start with a large amount of my own writing up front, thinking through the idea, listing several alternative ideas, asking a lot of questions, jotting down related topics, etc. and then reading the resulting output and critiquing it, asking for clarification, doing my own research on it, even just discussing it with the AI and doing this for a number of turns until I feel like I've exhausted the topic in the chat. I then often take what I've shaped in my own mind from that process and write something myself, either that or take the best parts of the output and edit, rearrange, reinterpret, and add to it in order to produce something.
I guess it all comes down to whether you are actively engaging with the material, regardless if that material is the result of a Google search, pulled from a book or generated by AI. If you just read it or copy paste it and don't engage with it and think about it then it doesn't do much good.