Posted by pseudolus 5 days ago
And for enterprise, they have many tools to scan vulnerability and malicious code before going to production.
Galaxy brain: just put all the effort from developing those LLMs into writing better code
They start out talking about how scary and pernicious this is, and then it turns out to be… adding a script tag to an html file? Come on, as if you wouldn’t spot that immediately?
What I’m actually curious about now is - if I saw that, and I asked the LLM why it added the JavaScript file, what would it tell me? Would I be able to deduce the hidden instructions in the rules file?
1. a dev may be using AI and nobody knows, and they are trusted more than AI, thus their code does not get as good a review as AI code would.
2. People review code all the time and subtle bugs creep in. It is not a defense against bugs creeping in that people review code. If it were there would be no bugs in organizations that review code.
3. people may not review or look only for a second based on it's a small ticket. They just changed dependencies!
more examples left up to reader's imagination.
This is a dystopian nightmare in the making.
At some point only a very few select people will actually understand enough programming, and they will be prosecuted by the powers that be.
AI generated code will get to production if you don’t pay people to give a fuck about it or hire people who don’t give a fuck.
You still have to review AI generated code, and with a higher level of attention than you do most code reviews for your peer developers. That requires someone who understands programming, software design, etc.
You still have to test the code. Even if AI generates perfect code, you still need some kind of QA shop.
Basically you're paying for the same people to do similar work to what they do now, but now you also paying for an enterprise license to your LLM provider of choice.
Literally all I’ve seen is stuff that I wouldn’t ship in a million years because of the potential reputational damage to our business.
And I get told a lot by people who really have no idea what they are doing clearly that it’s actually good.
Job security you know?
preprocess any input to agents by restricting them to a set of visible characters / filtering out suspicious ones