Posted by stevekrouse 4/14/2025
> cron job which makes a call to the Claude API
I am wondering, how powerful the AI model need to be to power this app?
Would a selfhosted Llama-3.2-1B, Qwen2.5-0.5B or Qwen2.5-1.5B on a phone be enough?
Also models have subtle differences, for example, I found Qwen2.5:0.5B to be more obedient(prompt respecting) and smart, compared to LLama3.2:1B. Gemma3:1B seems to be more efficient but despite heavy prompting, tends to be verbose and fails at formatted response by injecting some odd emoji or remark before/after the desired output.
In summary, Qwen2.5:1.5B and LLama3.2:3B were the weakest model which were more useful and also includes tools support(Gemma does not understand tools yet).
> I’ll use fake data throughout this post, beacuse our actual updates contain private information
but then later:
> which makes a call to the Claude API
I guess we have different ideas of privacy
I've also been following Anthropic pretty closely for the last two years and I've seen no evidence that they would break their principles here and plenty of evidence of how far they go to respect the privacy of their users: https://simonwillison.net/2024/Dec/12/clio/
I don't see what that is supposed to mean. What does that give you?
The alternative is either not using this stuff at all or restricting myself to the much less capable local models.
I don't take that competitive advantage particularly seriously, which is why I invest so much effort giving away what I've learned along the way for free.
How? This post shows nothing of the sort.
"I’ve written before about how the endgame for AI-driven personal software isn’t more app silos, it’s small tools operating on a shared pool of context about our lives."
Yes, probably, so now is the time to resist and refuse to open ourselves up to unprecedented degrees of vulnerability towards the state and corporations. Doing it voluntarily while it is still rather cheap is a bad idea.