AI agents: Less capability, more reliability, please

Posted by serjester 3/31/2025

AI agents: Less capability, more reliability, please(www.sergey.fyi)

423 points | 254 commentspage 6

donfotto 3/31/2025|

> choosing a small number of tasks to execute exceptionally well

And that is the Unix philosophy

bobosha 4/1/2025||

i think this agents vs workflow is a false dichotomy. A workflow - at least as I understand it - is the atomic unit of an agent i.e an agent stitches workflow(s) together.

segh 3/31/2025||

Lots of people are building on the edge of current AI capabilities, where things don't quite work, because in 6 months when the AI labs release a more capable model, you will just be able to plug it in and have it work consistently.

cube00 3/31/2025||

> because in 6 months when the AI labs release a more capable model

How many years do we have to keep hearing this line? ChatGPT is two years old and still can't be relied on.

postexitus 3/31/2025|||

and where is that product that was developed on the edge of current AI capabilities and now with latest AI model plugged in it's suddenly working consistently? All I am seeing is models getting better and better in generating videos of spaghetti eating movie stars.

alltoowell 3/31/2025|||

They're coming. I've seen the observability tools try to do this but I still have to tweak it. it's just time-consuming. Empromptu.ai is the closest to solving this problem. They are the only ones that have a library that you install in your to do system optimization, evals, for accuracy in real-time.

segh 3/31/2025|||

For me, they have come from the AI labs themselves. I have been impressed with Claude Code and OpenAI's Deep Research.

vslira 3/31/2025||

while i'm bullish on AI capabilities, that is not a very optimistic observation for developers building on top of it

techpineapple 3/31/2025||

In 6 months when FSD is completed, and we get robots in every home? I suspect we keep adding features, because reliability is hard. I do not know what heuristic you would be looking to conclude that this problem will eventually be solved by current AI paradigms.

thornewolf 3/31/2025||

GP comment is what has already happened "every 6 months" multiple times

techblaze3 4/1/2025||

Appreciate the effort in writing this.

daxfohl 3/31/2025||

We can barely make deterministic distributed services reliable. And microservices now have a bad reputation for being expensive distributed spaghetti. I'm not holding my breath for distributed AI agents to be a thing.

asdev 3/31/2025||

want reliability? build automation instead of using non deterministic models to complete tasks

revskill 4/1/2025||

Ai can uhnderstand its output.

anishpalakurT 3/31/2025||

Check out BAML at boundaryml.com

alltoowell 3/31/2025|

baml isn't great. You need to write it in their format and it still doesnt really solve the accuracy problem from humans interacting with your system that we're talking about here.

fennecbutt 7 days ago||

Lmao, training models off what is essentially a process directly inspired by imperfect "Good enough" biological processes and expecting it to be a calculator.

Ofc I'm not defending all thy hype and I look forward to more advanced models that get it right more often.

But I do laugh at him tech people and managers who expect ml based on an analog process to be sterile and clean like a digital environs.

aucisson_masque 4/1/2025|

Can you actually make the LLM more reliable tho ?

As far as I know, llm hallucinations are inherent to them and will never be completely removed. If I book a flight, i want 100,0% reliability, Not 99% ( which we are still far away today).

People got to take llm for what they are, good bullshiter, awesome to translate text or reformulate words but it's not designed to have thought or be an alternate secretary. Merely a secretary tool.

More comments...