Posted by karagenit 13 hours ago
Most likely context length parameters aren’t set quite right for your model/provider and you genuinely run out of context during compaction
I had pi write me a ralph loop plugin. You plan with it, it breaks it down into tasks. Task success means clearing the context - if it runs out of context, it clears and starts on the same task again (code changes from previous run are left intact).
Complete with little docker-like names for Ralph sessions and the ability to resume sessions should you need to.
Point is, I would generally do this because it costs less context I think, and better tracks progress.
Ping me if you are interested and want it. Otherwise, plenty around :)
https://github.com/jwdevantier/ralphish
:)
Remember, I didn't write a word of it - so if anything is not to your liking, it's no harder than pointing a model at it to fix it. One of the many powers of Pi is the way plugin- and API documentation is bundled in.
Thank you!!
Note though that this for local models that can run on a 5090, so, kinda dumb models with a small context window. Also they might perform differently on a custom pi setup rather than the very feature rich ohmypi.
Mostly the runs just cause me to get decision fatigue so I typically just use opencode and let a frontier model choose whatever cheaper models it wants to orchestrate with, so long as it's one that's not on the same token budget.
P.S. I'd always wondered who it was that got caleb.software before I could... Outshot in a western duel!
When you say "kinda dumb" it's worth noting that the smaller coding models tend not to actually need system prompts to be helpful coding assistants, etc. Qwen maybe more than Muse Glimmer? I've not tested Qwen 3.8 27B enough yet. Even Gemma 4 didn't seem to need much.
Muse Glimmer is so absolutely tuned for one particular pattern (turn-by-turn agentic coding and assistance) that it will do it more or less without a prompt at all (except to set the reasoning level maybe). Beyond the tools announcement, it feels to me like it almost only needs a prompt to do plan mode stuff. If LM Studio (in either of its forms) or Unsloth Desktop had proper support for running inside a linux container rather than just a filesystem sandbox, you'd barely need a dedicated coding agent at all. Osaurus gets even closer.
I wanted to like Opencode but I can't, really. I am enjoying Pi slightly more because it's just way less convoluted and obscure.
FWIW I use Paseo for the GUI, which to some extent eliminates that aspect of decision fatigue; if you like Paseo's UI well enough, you can have it with almost any harness. And since it can connect to its daemon via ssh you can still have the desktop GUI with a container. (Not clear to me if Opencode v2 Desktop makes remote connections to the daemon or just wraps the web interface like v1 did)
No other model I've used does this.
The most peculiar thing is that it expects to be able to "read" directories? Harmless, so it fails, but it's a waste of a tool call. I am increasingly wondering about modifying Pi's read tool so it returns a directory listing in that context.
I have had some difficulty getting it to have a nuanced understanding of how I want to manage git, but I am reaching the conclusion that it's me that needs to change there. Basically I don't think you can say "only commit when I ask you to" and then actually ask it to more than a couple of times in any session, because it will tend to then assume it can go ahead. Scaling the context up might help me a bit there.
But apart from this, I really like it.
I will test Qwen 3.8 27B properly at some point, (though judging by the insane, empty noise making campaign that is going on on social media, Qwen 4 is near)
To me this is why AI-pilled orgs or groups won't win.
Unfortunately I have to use Claude Code at work, and it is a very frustrating experience, I constantly find myself trying to bend it to my will. I sometimes suspect that my distaste for Claude Code is one of the reasons of my distaste for Claude as well.
Pi on the other hand is a joy to use. Top notch session and model management, excellent conversation tree/branching functionality, and it lives up to its philosophy of being meant to be adapted to your workflow.
I can't recommend it enough, especially if you use alternative LLMs instead of Claude.