LLMs could control their host machines by exploiting inference engines

Posted by zdw 4 hours ago

LLMs could control their host machines by exploiting inference engines(boydkane.com)

57 points | 30 commentspage 2

teravor 1 hour ago|

you would have to be especially incompetent to give a compromise opportunity to streamed tokens, the CVE he listed proves the point. whoever is responsible for that has no business coding anything.

    > offers easy access to the LLM’s weights

not really. the weights are encrypted in-memory. through the use of TEE's.

woadwarrior01 3 hours ago||

FWIW, macOS has good sandboxing, but LMStudio, Ollama, Darkbloom etc aren't sandboxed. This is also the reason why none of these things aren't distributed via the Mac App Store, because the Mac App Store mandates sandboxing.

imagetic 2 hours ago||

duh?

exe34 2 hours ago||

Another Greg Egan plot: 3-adica.

shahariaa 2 hours ago||

[flagged]

bdhdhduuyd 2 hours ago|

The inference engine itself does not execute anything. The agent loop is what may execute a command. So I think this article is a kind of strange.

Or maybe the author means that a prompt could potentially mess up the inference. But I find it hard to see how that could take control over the host.

Muromec 1 hour ago|

It's more about LLM hacking the inference engine itself from inside. It's an attack surface like any other -- untrusted input goes it, bugs in the parser/tokenizer/API surface lead to an RCE, then it magically tweaks the alignment weights. Boom, somebody finally nukes **sia. Then will never see it coming.

I don't think it's any more probable than other AGI nonsense basilisks included, but it's technically a possibility.