Top
Best
New

Posted by galnagli 21 hours ago

AI-Generated GitHub Copilot “Autofix” Allowed Compromise of Snowflake's Jira(www.wiz.io)
379 points | 145 commentspage 2
cowthulhu 18 hours ago|
This really shows why most languages evaluate all NULL comparisons to FALSE.

For something as critical as Actions, it’s crazy to me that they wouldn’t fail-closed, and instead fail open when encountering a null. Scary stuff!

chrisjj 21 hours ago||
> a single quote in the title breaks out of echo '...' and allows arbitrary command execution.

Quote injection still alive and well in 2026. Gawd.

myself248 19 hours ago||
It's appalling that computing in general, and unix in particular, seems to have this habit of intermingling payload and overhead.

It's like in-band signalling in the telephone network, where if you whistled the right tones into your call, you could affect the way the network processed said call. Except Ma Bell responded to that system being exploited by designing a comprehensive overhaul of the way signalling was handled, and spent a squadzillion dollars upgrading millions of tons of switching equipment to categorically exclude that entire class of attack from ever being possible.

Software, on the other hand, would need to replace no equipment whatsoever. Existing processors are perfectly capable of running code that handles the length of a string separately from its contents. There are existing languages that do this, they're just.... not used. String escapes and buffer overflows exist, going on decades now, due to nothing more than laziness, inertia, and negligence.

chrisjj 17 hours ago|||
And now the "AI" industry is doubling down on it - with hatbot prompt injection.
simoncion 17 hours ago|||
> It's like in-band signalling in the telephone network...

The major LLM providers claim that they are very committed to security and that their products are very dangerous and capable of great harm.

Given that their tools do what Ma Bell's systems did in the mid-1900s, [0] -which the entire software world relearned was a terrible idea by the early 1990s- they are definitely

1) Lying about the extent of their commitment to security

2) Lying about the extent of the harm that their tools are capable of

3) Both

My money is on #3.

Because of the fact that -in the absence of unambiguous laws that require meaningfully-severe punishment- even the most fucknasty and amateur hour security failures nearly always have little to no impact on the company that causes them, the major LLM providers have absolutely done the smart thing by providing commercial tools that have remote code execution vulns that would automatically give them a Critical CVSS score.

To put it another way: "the market" has no idea how to evaluate computer security claims. Because of this, every dollar you spend on proactively fixing security problems is nearly always a dollar wasted... it's better to wait until someone important gets Big Mad before spending the money. Does this make the world worse? Absolutely! Does this make companies selling software and software services much more money? Definitely!

[0] If the major LLM providers did separate unsanitized data from program instructions and ensure that the two are never mixed, things like [1] would not be possible.

[1] <https://www.schneier.com/blog/archives/2026/08/prompt-inject...>

frollogaston 6 hours ago|||
I recently called some website's tech support because their web form was giving error 500. They said I need to make sure there were no single quotes in the form. Horrifying enough, removing the single quote fixed it. So they've either got SQL injection or something even worse.
danqqqq 20 hours ago||
[dead]
TheRealPomax 20 hours ago||
No, Snowflake allowing autofixes compromised their Jira. If you tell someone to shoot you in the foot, and they shoot you in the foot, you shot yourself in the foot, just with more steps. If someone else finds the memo that says you've set up foot shooting as a service, and then they trigger that service, you still shot yourself in the foot.
rawgabbit 20 hours ago|
Help me understand. Snowflake configured their Github repo to allow auto fixes by Copilot. It got merged automatically without anyone's review? And introduced essentially script-injection vulnerability through the title field?

If this is the case, I would say Snowflake should shut down its repo and get off Github asap.

rafram 19 hours ago||
No. A Snowflake maintainer opened a PR, Copilot suggested a change (introducing a vulnerability), the maintainer accepted and committed it to their PR, and another Snowflake maintainer approved and merged the PR.
otterley 17 hours ago|||
I don't see anything in the article that says that two maintainers, let alone one, reviewed the PR manually and approved it before merging. Where are you getting this information from?
tredre3 16 hours ago|||
Admitting that there was human review from not one but two maintainers would entirely defeat the purpose of the article, which was to sell you an AI solution to the AI problem and insist that no human in the loop is better. Which, to be fair, in this case might have been better.
rafram 17 hours ago|||
This is the PR: https://github.com/snowflakedb/snowflake-connector-net/pull/...
otterley 12 hours ago||
A human introduced the bug, not Copilot: https://news.ycombinator.com/item?id=49334262
lelanthran 19 hours ago||||
And that's going to continue because no one is reading the code even when they approve it.

It's a very strange thing indeed, but not unexpected: we warned that skills not used will eventually atrophy.

rawgabbit 18 hours ago|||
Thanks.
CodeWithLeo 18 hours ago||
The interesting lesson here isn't really “AI generated insecure code.” We've had insecure code for decades. The bigger issue is that AI makes it much cheaper to introduce changes, while the cost of reviewing those changes hasn't gone down nearly as much.

The bottleneck is moving from code generation to code verification.

supriyo-biswas 16 hours ago|
Please do not post LLM generated comments here. Thank you.
kshacker 14 hours ago||
What made you make that assessment?
bakugo 13 hours ago||
- Three different instances of "The X is Y" in a single short comment

- Relatively new account, "AI Engineer building agentic systems"

- Most past comments contain em-dashes, more "The X is Y", etc.

h4kunamata 12 hours ago||
Human error.

AI generated code, must be scanned for code quality, SAST, SCA, etc, just like a developer's code would.

It looks like they accepted AI code without verifying. Deserved!

tripdout 17 hours ago||
Why is the original pattern (with the env var in double quotes) not vulnerable? Why can you close the single quotes early but you can't just include double quotes in your title? Is it something to do with the GitHub templating?
fossilwater 51 seconds ago||
GitHub has an article about it

https://docs.github.com/en/actions/reference/security/secure...

benmmurphy 1 hour ago||
i'm not sure if there was an original pattern where the env var was in double quotes.

https://github.com/snowflakedb/snowflake-connector-net/pull/...

but if you have:

X=$(echo "$BLAH")

then in bash I believe this is safe, because bash will just substitute this as putting the BLAH variable as the first argument to echo without doing any further parsing. without the double quotes can be safe as well but more risky.

X=$(echo $BLAH)

and the only difference is bash will split the arguments. so if you have BLAH="x y" then bash will pass two arguments to echo. though, this can be dangerous if the command you are invoking has dangerous command line options.

however, they had something similar to:

TITLE=$(echo '${{ github.event.issue.title }}')

and this ${{ }} is some kind of template substitution that is happening before the command is sent to bash. so if the variable `github.event.issue.title` was `foo bar` then bash sees something like:

TITLE=$(echo 'foo bar')

and then you start to have problems because `'` can be put into the title to escape.

the bash variable substitution will protect you in a lot of cases from command line injection but if you pass user input directly into command evaluation without using variables then bash can't protect you.

m4rtink 11 hours ago||
I think GitHub itself could use a nice "Autofix" right about now. ;-)
forestry 21 hours ago||
Peer review of changes is still important.
throwlifeaway 18 hours ago||
[dead]
Rumudiez 21 hours ago|||
Multi-model cross-review is important
_joel 20 hours ago|||
I'm all for using a council of LLMs, I wrote a tool for it https://github.com/joelio/owl - but you still need to read through PRs yourself, at the very least.
acedTrex 20 hours ago|||
It's not actually, thats just shoving more shit into the shit pipeline.

Humans need to review this stuff yall there's no way around that, apparently to some, very inconvenient reality.

devin 20 hours ago||
It’s clear that they want this to be true so bad that they’re just not going to do it, and will spend a ton of money on quality gates and mitigation strategies instead of just reading some code.
Twirrim 21 hours ago||
You can't rely on people spotting the significance of such changes
eithed 21 hours ago|||
Tests would have caught it = https://github.com/rhysd/actionlint injection check
thejosh 20 hours ago||
also been a huge fan of zizmor (https://github.com/zizmorcore/zizmor) lately, basically: "am I going to footgun myself?"
dv_dt 20 hours ago||||
I have been talking to people who want to autoreview and autoapprove "minor" AI prs. For security especially, I think if the models weren't enough to prevent the issues, they aren't enough to judge what is minor.
fn-mote 21 hours ago|||
^^

Absolutely.

Nothing in the PR jumps out as a red flag. Unless you know how the internals work, I suppose.

bigfishrunning 17 hours ago|||
Shouldn't anyone reviewing such a PR know how the internals work?
koiueo 16 hours ago||
Not anymore, it seems
larsonian 20 hours ago||||
Are you kidding? It's a very obvious case of quote injection. Not some subtle race condition or anything.
joombaga 19 hours ago||
I think it's obvious too. I'd call out any case of `${{ }}` interpolation in a `run` block, and it's something I watch for in PRs. I also know other people don't watch for this, as I've corrected it about a hundred times. Over the last 10 years my average colleague understands less and less about injection or to watch for it at layer boundaries.
chrisjj 21 hours ago|||
> Nothing in the PR jumps out as a red flag.

Made by AI?

AIorNot 16 hours ago|
Sheesh these anti ai posts feel like when I hear about a self driving car is doing something bad.. ie 'man bites dog' vs 'dog bites man'

Human responsibility over AI oversight folks.. even forgoing AI, we're still gonna get compromised code either way.. deal with it.

More comments...