Top
Best
New

Posted by davidest 3 hours ago

Ask HN: What is one simple thing LLMs are insanely bad at?

I am looking for ideas on what to train a specialized model for!

What is one simple thing you repeatedly ask ChatGPT, Claude, or another model to do that it still somehow messes up?

25 points | 51 commentspage 2
humanrebar 2 hours ago|
Short answers to simple questions.
honr 2 hours ago||
Accurate short answers / text are always harder than long answers, for human or AI. I know several authors and editors who write a lot longer at first, then spend a multiple of the initial time compressing it via a back and forth process to something dense. Sort of like weaving the initial threads.

I found this can work with AI. You get it to generate a lot more at first, and then do several passes over it to compress and squeeze out the noise while keeping the core information. With AI, at least with my prompts, it takes some effort (on my end) to get it to really really cut down the noise and not cut everything out.

FriedFishes 2 hours ago||
If I had more time I'd write a shorter letter, a la Pascal.

Editing is generally hard work, at the current token price I don't mind spending multiple passes of high effort to get down to a reasonable noise/signal ratio. I've seen some people pass off output to a weaker/cheaper model but that makes me a bit nervous when I don't have intimate knowledge of the subject.

nunez 1 hour ago||
I've had some success with adding something like "answer in no more than n words" to the prompt
znnajdla 2 hours ago||
Editing a document without mixing edit instructions into the final document. Claude and ChatGPT do this all the time: I tell them to change X in a planning document or email draft, and instead of just changing X they also frequently add the edit instruction to “change X” into the document itself. They seem unable to take a step back and look at the document without “becoming” the document somehow. I do believe that dedicated subagents for editing may fix this but I am not sure.
spike021 2 hours ago||
I've had a lot of trouble when it comes to sorting out UIs. I've tried with an iOS game and also a TypeScript app with UI elements from libraries like ReactFlow. The usual models can sometimes fix or change things based on screenshots but more often than not they just don't "get it" (e.g. certain shapes on a plane are overlapping, which I don't want, the models can't fix what they can't "see").

I've had some luck on the web app side if I use playwright or similar for the model to interact with but still far from efficient.

newsomix9xl 2 hours ago||
Picking a random number between 1 and 30.
sghiassy 2 hours ago||
Generate an image of an analog watch with its hands set to the time specified by the user

More of an image model than a LLM model tho

alexandra_au 2 hours ago||
Being able to read and translate Egyptian hieroglyphs. You may think this is silly but a trained LLM to translate hieroglyphs would be amazing.
alexeldeib 2 hours ago||
What issues do you see in practice? This seems pretty easily "fixable"
krapp 1 hour ago||
>You may think this is silly but a trained LLM to translate hieroglyphs would be amazing

Why would it be amazing? We've known how to read hieroglyphs for a long time. It isn't a problem we need computers to solve.

dowonseo 1 hour ago||
Something creative and not normal. like ideas
dSebastien 1 hour ago||
Counting things
dhruv3006 2 hours ago||
Its extremely bad with Sign Language,Fact Verification.
TZubiri 3 hours ago|
Suggesting business names for businesses, I mean they are great, but they already exist, multiple times even.
mojuba 1 hour ago|
True, tried it so many times and every time I come up with something myself though sometimes inspired by the AI's ideas.

Verifying trademarks and domain name availability is usually an additional step you need to ask it to perform. Trademark DB searches by the way are intentionally made difficult to scrape so most of the time it's a manual process anyway.

However, once you give it all the information (TM search results, domain name availability) it can help you with the judgement of how safe the name is from the legal perspective. With the obvious caveats, but still a good starting point if you are serious about the name.

More comments...