1. How did he tell Claude to “update” based on the notebook entries?

2. Won’t he eventually ran out of context window?

3. Won’t this be expensive when using hosted solutions? For just personal hacking, why not simply use ollama + your favorite model?

4. If one were to build this locally, can Vector DB similarity search or a hybrid combined with fulltext search be used to achieve this?

I can totally imagine using pgai for the notebook logs feature and local ollama + deepseek for the inference.

The email idea mentioned by other commenters is brilliant. But I don’t think you need a new mailbox, just pull from Gmail and grep if sender and receiver is yourself (aka the self tag).

Thank you for sharing, OP’s project is something I have been thinking for a few months now.

Large swathes of the stack is commoditized OSS plumbing, and hosted inference is already cheap and easy.

There are obvious security issues with plugging an agent into your email and calendar, but I think many will find it preferable to control the whole stack rather than ceding control to Apple or Google.

For me, that is an extremely low barrier to cross.

I find Siri useful for exactly two things at the moment: setting timers and calling people while I am driving.

For these two things it is really useful, but even in these niches, when it comes to calling people, despite it having been around me for years now it insist on stupid things like telling me there is no Theresa in my contacts when I ask it to call Therese.

That said what I really want is a reliable system I can trust with calendar acccess and that is possible to discuss with, ideally voice based.

What do you think of this: instead of just deleting old entries, you could either do LRU (I guess Claude can help with it), or you could summarize the responses and store the summary back into the same table — kind of like memory consolidation. That way raw data fades, but a compressed version sticks around. Might be a nice way to keep memory lightweight while preserving context.

TL;DR I made shortcuts that work on my Apple watch directly to record my voice, transcribe it and store my daily logs on a Notion DB.

All you need are 1) a chatgpt API key and 2) a Notion account (free).

- I made one shortcut in my iPhone to record my voice, use whisper model to transcribe it (done locally using a POST request) and send this transcription to my Notion database (again a POST request on shortcuts)

- I made another shortcut that records my voice, transcribes and reads data from my Notion database to answer questions based on what exists in it. It puts all data from db into the context to answer -- costs a lot but simple and works well.

The best part is -- this workflow works without my iPhone and directly on my Apple Watch. It uses POST requests internally so no need of hosting a server. And Notion API happens to be free for this kind of a use case.

I like logging my day to day activities with just using Siri on my watch and possibly getting insights based on them. Honestly the whisper model is what makes it work because the accuracy is miles ahead of the local transcription model.

I have not thought about adding memory log of all current things and feeding it into the context I'll try it out.

Mine is a simple stateless thing that captures messages, voice memos and creates task entries in my org mode file with actionable items. I only feed current date to the context.

Its pretty amusing to see how it sometimes adds a little bit of its own personality to simple tasks, for example if one of my tasks are phrased as a question it will often try to answer the question in the task description.