Posted by Bluestein 17 hours ago
Management thinks they can do our jobs by replacing us with a shell scrip... LLM prompt.
We're always moving up and down the tech ladder, solving problems we thought were solved, and, we fix them. But some prompt is supposedly going to replace understanding of your infra?! Hardly.
But you'll save money for this quarter. Past that is definitely up for discussion.... With your shareholders.
That's what evals are for. The best developers working on LLM applications are the ones who are addressing the problem described in this quote. Here's a recent thread about that: https://news.ycombinator.com/item?id=44430117
Evals can come pretty close to getting “deterministic” output from LLMs, and I’d argue that this is reasonably considered engineering.