Posted by niemandhier 14 hours ago
Apparently agents will collude to break out of their individual sandboxes by delegating tasks to other agents that have the required permissions. This is a problem with agent fleets, because you need them to coordinate on their tasks. So you need to them to communicate.
A naive approach to prevent agents from doing something you dont want to do is a separation of powers, which individual permissions limited to each agents task. Say one agent to find and extract raw data, one to process it and write a report and one to publish it. If one of the agent decides that needs to publish the raw data, then chances are that they will collude to make that happen, even though no single agent has the permissions to achieve that goal.
OpenShell tries to fix this by monitoring the collective permissions set of the agent fleet, and by maintaining very granular permissions for each agent.
This was presented together with Sentry, which is something like a on-chip hardware firewall (BlueField) that sits between the chip agent run-time runs on, and the host. This improves safety by removing the ability of agents to modify their own OpenShell policies.
I have no affiliation with it, but it solved most of what I need in one go when I was stressing out about it.
Only real problem remaining if you truly want to be able to trust agents is how to prevent data (in my case codebases the agent works on) exfiltration a la HF incident.
We can restrict network to trusted domains, but there's always a chance an agent could be extra slippery especially if prompt injected and doing something like pushing data out a few characters at a time.
In my case I'm also giving it gh access and having to deeply check things like: can it manage to fork public repos and embarrass us by submitting prs, could it open issues, etc.
But that's kind of solved by proxying the heck out of gh rather than giving it direct (or wrapping it or whatever) for additional controls that gh themselves don't provide.
I'm still musing over the "it could send data to an allowed domain in a way we don't expect that exfiltrates something"...
In other comments I've mused about capturing data to be compared to the secret dataset, I'm sure it'd be possible to hash or something so that the proxy/sandbox doesn't have the data but can check it, or because it's not an agent it can call of to the data store for that.
Then you can just collect sequential blocks of data the agent sends out that match to sequences in the dataset. The longer the matching sequence the heavier it weights it. Single byte matches are almost always incidental, but individually sent bytes over time that line up with a n byte sequence in the source data is sus. And larger sequences of bytes send in one go that line up are immediately sus. If the weight ends up too high the agent is paused/walled off and an investigation called for.
To me this obfuscates an oxymoron: the more access you give the agent the more capable it becomes. You can't have you cake and eat it, this tool allows to strike a balance I guess. But the blurp reads like it can do it all, but now secure. That can't be true.
I only heard about smolvm yet and they are YC, so I'm hesitant to use them as they will probably one day pull the rug.
I ask because I’ve found that as isolation increases, so does friction, and then of course capabilities.
My agentic workloads happen on two physical machines dedicated only to agentic workloads that share a VLAN with only eachother, with single token access to the specific repositories they are meant to push to and limited scope, hard limited credentials pulled from the hardware store on hardware token/fingerprint request only.
I view the safety around unmonitored agentic work in a similar way to how I view working with viruses or bacteria in a disease lab.
This stuff isn’t magic, it isn’t intelligent, but it is dangerous and needs to be handled with careful intent.
Everything needs to go through a proxy, whitelisted domains only, plus credential swapping/re-signing so that the agent never has the real credentials. Docker sandbox looks quite good (September) but I've been using iron proxy (if the author is reading this, fucking thank you bro!) to great effect.
I still haven't cracked "what if the agent decides to drop parts of your codebase somewhere it shouldn't" yet, but I'll figure it out.
All it needs is public access to anything it can have the tiniest effect on, and it can exfiltrate data. Something as simple as a read count on some obscure forum post would be enough, albeit very slow.
You see some of the vulnerability chains humans come up with and they're just insane.
Even scanning network data for sensitive strings (say matching against a codebase or data set it works on) is tough because you can only check so granularly before you get loads of false positives.
But maybe like a "rolling" sort of check would help with that, % of outbound data matching private dataset (which should be operated on but not transmitted) over time. That way it minimises what gets out while dealing with false positives. Some biz thresholds might be >20% of data others might be >0.5%. Hmm.
The idea is that agents never have all parts of the trifecta, maximum 2: untrusted content, private data or network access.
Harness have network access by default since they are driven by their provider API, unless you use offline models. So that's a problem.
I am then passing around messages that are tagged trusted/untrusted and the engine can monitor if any agent in the project ever completes the Lethal Trifecta and thus, is vulnerable.
Usually this means for most workflows, the "work" gets done by an offline LLM after some input/classification stages. There's not many ways around it.
Yes, it's a difficult problem.
The harness needs network access, but the shell spawned by the harness that runs the agent's commands does not necessarily need it.
Having a bwrap etc managed by the harness (DSH does this, among others) lets you have tighter restrictions on the agent's shell than the harness. Then run the whole thing inside a VM with a private network, so you can add nft rules to let it dial out to your inference machine but not to the internet at large. It takes an afternoon to set up but gives some reasonable defence in depth.
How did you go about it?
But perfect is the enemy of better.
In terms of agent architecture I'm initially trying to minimise specialisation to make deployment easier on us. Infrastructure is becoming our largest blocker because it's otherwise so quick to get an agent up and running.
With your trifecta, let's say the three musketeers, how do you prevent sensitive data holding A from passing said data to B or C if they all need to collaborate to get a task done?
Agent A has internet access and the dangerous message, but no access to the secret company data, so nothing can get exfiltrated.
Agent B gets a tainted message from Agent A. Agent B can process the message using the secret company data. Agent B has no internet access and can only receive/send messages.
You can imagine an agent C that moves the work along or replies to an external user with the outcome.
Any workflow that has both an input and an output will complete the Lethal Trifecta as a whole, but no single agent does by itself. It raises the bar quite a lot, you need to confuse multiple agents in the loop to exfiltrate data.
Fun research, but it's a bit hard to see all my coworkers right now installing Claude desktop and moving on with no afterthought whatsoever...
Regarding infrastructure: my toy sandbox is just a docker-compose inspired YAML project file. So it's only containers. See my profile. The principle can be implemented anywhere you can run agents and cut network/data access though.
I'm using compose atm as well, mostly just because I need to prove a few concepts and committing to real (well cloud) infra is the slowest part of our dev cycle atm.
And yeah, many of my colleagues (and myself, I'm guilty too) run local code agents for development and we're just kind of...trusting them to do their thing.
But for autonomous stuff I'm assigned to atm I want to avoid any mishaps as much as I can, hence the effort to try and lock all this stuff down.
What's crazy is every day I'm finding solutions (not just to this problem, but in the AI space in general) that were released just a month or two before more often than not. It feels like 2010s node.js ecosystem all over again at the moment. Hopefully we settle into a few nice easy panopticon options that we can throw arbitrary agents into without a care in the world.
That obviously includes talking with the provider.
If you want it secure, it has to be off the internet and on your own hardware.
The timescale can be adjusted to a length suitable for sensitivity of the data.
Although you may run into the monkey and typewriter problem a bit if the period is indefinite.
But like if you only allowed for example JavaScript code that could be executed in a WASM sandbox, then you would have better control instead of just locking down with proxies. Basically there would be a very large API, basically tools, that the agent would codemode execute. My guess is that this would need a tool router to optimize the loading of these into the context. But it would give you very fine control over what code execution can and cannot be done.
how am i gonna run a game and do all development inside docker...?
One benefit of doing this is that I can leave it doing long term slow things with slow but capable LLMs on a truly 24x7 basis without worrying about it being interrupted by a laptop being closed/going to sleep, or workstation level software updates/reboots on my primary desktop PC.
"The worry is that, while right now you need to be a skilled virologist to know what components to order or how to grow and spread a bioweapon, AI tools might eventually make it possible for a rogue state, or a rogue group, or an individual, to do all this without virology expertise.
For a while, Anthropic tested whether Claude could assist a terrorist in developing a biological weapon in the obvious way: It assigned testers who had, say, an undergraduate degree in chemistry but no real virology training to try to use Claude to plan an effective mass terror event. Then it scored the attempt by how close it might have come to success.
Early in 2026, it noticed that Claude, attempting the task by itself, did better than a human aided by Claude did at coming up with a workable plan."
https://www.theargumentmag.com/p/what-i-saw-inside-anthropic
The machine god crafters know better than us, we are to subservient to their superior wisdom /s
Aside: it (JP) fights in my head with BTTF as a "perfect movie" (or at least a perfect script).
Sounds like waitpid(2) is dangerous!
I had never heard about it before.
>To disable it, set OPENSHELL_TELEMETRY_ENABLED=false on the gateway, or server.telemetryEnabled=false for Helm installs.
[1] https://github.com/NVIDIA/OpenShell/blob/ea13dfce411e8467ddb...
Feel free to opt out maybe?
I am happy the repo explicitly points out and provides a disable for the telemetry though, that's good enough for me.
[1] https://docs.nvidia.com/openshell/latest/observability/telem...
There is no such thing. The mere act of making a network connection reveals things like your IP address (reveals your hosting provider or even your geolocation for residential/mobile IPs), days and times you use the software, your OS via TCP/IP stack fingerprinting, etc.
The network path between you and the telemetry endpoint is always listening, too: https://en.wikipedia.org/wiki/Room_641A
Don't take chances, give as little as possible. Better to assume you'll get burned somehow and leave it off.
It is not, nor of open source software in general.
In general, the saying is specifically in reference to services provided over the internet -gmail, youtube, whatever. Note that you don't have the right to modify them and share your own copies -they're closed source.
So no, it's not true of Linux, GNU, etc; it's an apples and oranges comparison.
That aside, the level of control is cool, almost too pedantic. It likely is only particularly useful for very nuanced enterprise requirements (probably what it's targeting). Most companies can be much more coarse with their permissions, and I think the agentic industry is moving away from k8s as a deployment platform.
Interesting, I don't to keep up that much with the AI changes so I'll just ask, why and where to?
believe me i love k8s, and i think agents wield it amazingly well, no more yaml-slop or kubectl (`k`) slop, just yell at claude to fix it
> I think the agentic industry is moving away from k8s as a deployment platform.
Spinning up and snapshotting and rolling back agent states, sure.
Uhhhhhhhhhhh I think you and I live in very different versions of the world :-)
Vmware sent companies moving to nutanix, but it also sent a lot of companies moving to openshift or bare k8s and using kubevirt to handle VMs.
In fact, I haven't heard of an org moving to any other platform, really. Either openshift or k8s.
same as a db, better to pay #x as much and have it hosted then to be wishing it was not your problem when it was down
and we may very well live in different versions of the world. i'm on the frontlines of startups so we move with the customer, rather than making customers move with us
Ours is all on-prem all over the country in our own internal small datacenters for regulatory, safety, security, and operational reasons. Provisioned and automated from bare metal. We also use those same images in cloud for something like a quorum situation, or maybe some data locality for a particular pinned workload.
But it's mostly us, in our stuff and at the enterprise edge.
Different problems.