Posted by Anon84 2 days ago
The example listed in the article -- fanning out a few simple get-population, get-timezone, and make-summary calls -- is, in fact, useless overengineering. This is a basic promise chain with extra steps (priced with tokens).
But as with all software pattern learning, we learn the concepts with simple toy examples that generalize into something bigger. It's the generalization that matters here.
This is talking about a few methods and tricks for spawning effective subagents (collectively, that's the "harness"). Those tips and tricks are nice, but to not be considered useless, we need to make sure we understand why spawning subagents is useful in the first place. Yes parallelism is nice for some tasks, but that's not really what this is about.
The real reason is protecting your context. Yeah, we have 1M context windows that can fit all of LotR in it, but these machines work better when they're narrowly focused. Large context windows run into attention issues and forgetfulness ("Yes, you're right, it was stated I should/n't do X but I ignored it, my bad."). So subagents come into play when you don't want all the tokens associated with a subtask to pollute your main/primary context window and degrade task attention. Split that off to a subagent, let that context navigate the details, and just make sure your main one gets just the input/output blackbox results.
The trick is getting a sense for when the complexity of the task warrants that kind of context protection, vs when a single agent is good-enough. Your toy example will never have enough complexity to warrant the setup, but you might one day find a generalization that may.
I don't think they are totally usesless.
And its clear that progression is happening on a communith level on all of these and they get integrated later on in commercial offerings like from Anthropic and co.
But also doing a opensource harness and not just giing in to the big companies allows us to have all of this open and transparent and with open models locally.
But there is another aspect which I do enjoy which is closer to the feeling of dialing in key bindings in vim or getting a really good rhythm going with your vscode extensions or zhs plugins. It is that level of "I want my system to do exactly this thing in exactly this way" customization that a lot of technical people crave.
And you can do it with harness and context engineering in many cases. In other cases it introduces friction because it will be like "cool, I will only output 15 words max unless told otherwise" and then in the next turn completely disregards it with an "oops, you did tell me to do that didn't you."
And that frustration compounds when older model versions may have done a better job of that but new models are like "thank you for your suggestion, your opinion, while appreciated, is irrelevant. Now let me get back to overspending on your token budget. "
See what some guys like Linus Torvalds, or Eric S. Raymond are saying about. It's not so much about "vibes" but using the tool (yes the AI tool) in a certain way that can propel yourself towards your goal at unprecedented speeds.
I think most would agree with that, unfortunately, this is not how LLMs or agentic flows are being marketed, nor how firms are encouraging their users to use them. It's being sold in a manner completely different from its capabilities.
I've maintained this for a long time but if AI was being sold as a way to achieve superhuman levels of productivity and to help you with certain goals or tasks then people would be far less opposed to its adoption. The social havoc it's wreaked as a result of Altman et al.'s bluster about it being a fully ready, human-level replacement for humans has done its reputational damage and it will take years for that to recover, especially given that they don't seem to care (yet).
Trying to make gold from pyrite.
But pulling orchestration off is very very tricky.
Even if it is just a small, simple orchestrator.
Ideas like planner, memory, log, subagents, graphs (each on their own) sound great and very promising.
So promising that one would think they must work, how could they not?
I've been there as well!
The challenge is that all these parts of the orchestrator are intertwined with each other
and they are all causing overhead in the main context window in some form or at least overall complexity that is difficult to grasp and predict/engineer for
(even though the idea is to help exactly with the fact that the context window is limited)
To save context window there is also more communication that has 'stille post' ('chinese whispers') like dynamics
Turns out it is very difficult to find out the right context to bubble up and down.
It's very similar to human org communication challenges (think large org stucture vs small teams vs one person that can keep it all in their head)
Yeah, what do you do if one person can't keep it all in their head?
But how great is it when it's possible?
Companies must have figured out how that works right? Maybe we can adopt and implement these ideas?
And yet … easy it is not, especially when you're not dealing with run-of-the-mill well-defined tasks.
But more like with open-ended software development?
I'm not saying it's not possible or that it should not be tried.
On the contrary, I think this is worth pursuing and a bit like the search for the holy grail.
But I also think the other direction of the search space is under-explored.
The holy grail is glamorous.
With 'smol' I'm spelunking on this other extreme (non-orchestration?)
(welcome, join us, we have cookies, and context windows with a lot of room for work items!)
smol is a minimalist agent harness that protects the context window
- no system prompt
- no tool spamming (just 1 tool: sh)
- no agents.md
- no mcp
- no planning, todos, graphs, beads, …
it is a worthwhile thread to pull I think
at least from the dozens of benches I'm looking at I see that less stuff in the context window does help a lot
- cheaper per task
- finishing faster
- better tool composition (sh and pipes are great!)
the context window is precious