You can do that with smaller models at home. Gemma-4-E4B will run on a 12gb GPU, and supports audio, image, video input

just use one of the various cheap gemini models

Or you could just use a CNN...

If you spend your life sitting in a chair, that's fine. I tend to get all kinds of ideas, questions, and research needs while I'm walking around. Typing a paragraph or two or context takes too much time and is very risky. Especially when driving. But also just walking, cooking, cleaning, etc. Sometimes it's just not practical - winter, carrying stuff... I mostly feel privileged if I can just sit at a computer and type my question and have the time to read the answer.

It’s crucial to use for driving/walking.

One problem has been ChatGpt/Claude apps don’t really do this well. They use weak and/or non-reasoning models for voice interaction and the UX is not optimized for hands free.

I wrote an iOS chatbot app mainly for this purpose for myself and family/friends. Allows starting/sending voice prompts with the action button so I never have to look at the screen. Supports any model at any reasoning level so conversations are not dumbed down. Added a video transcription tool so any model can “read” YouTube/Tiktok videos and chat about them. Great to discuss lectures on tech topics.

It takes slightly longer to use a reasoning model for voice interaction use but I prefer the intelligence. The latency can be minimized a few ways, bidirectional streaming helps. It’s TTS agnostic, I’ve got a few selectable providers and the output can be prompt styled “use a chill tone that’s not too eager”.

I am someone that prefers a slack message to a coworker than talking to them and I use AI.

My current flow is: Google Eloquent to capture 127WPM (my typing is best case is 65wpm). This lets me get the thoughts out without thinking too much about structure or flow, the same way I would brain-dump type it.

Next I use AI to compress, summarize, and restructure to create a clear coherent message for my peer to read (which is way faster for them).

When communicating with AI, its the same thing, except I skip the second step since AI does a good job at understanding my ramblings.

----

It drives me crazy that some cultures only send voice messages to each other. It drives me crazy they can't be respectful of my time and use STT+AI to convert their 90 second monologue to a few written sentences.

I hardly type at all now. I use Handy (free) with Parakeet and use its post-LLM processing feature with a custom prompt tailored towards coding, so I can say things like "Have it go to slash remote dash control" and it'll output "/remote-control". Converts brackets, etc.

Everything is almost instant, it's insanely fast, and lets me work on multiple different agents/windows at the same time fast with cmux.

I use the same thing to talk to people on Slack, iMessage, etc now when I'm working from home instead of typing.

I also can help articulate my thoughts better when I'm thinking them literally out loud instead of just sitting silent and typing them on a computer for hours.

It's just something that you need to try and get used to because I also thought it was something I wouldn't like at first.

I thought this way until I tried it, and the main difference is that when I'm managing tons of agents at once or just reviewing some plan / approving next steps, or need to give quick feedback/ask a simple followup, the voice interface makes me much faster and more likely to continue because it's lower friction (and in many cases that's good, though not all) and can be hands-free.

Actually, my thoughts on this matter changed so much that it inspired me to get much more into voice controls because I realized how this same problem was basically why some people sucked at remote work or weren't able to properly use tools like claude code, because it was essentially the same problem but worse (typing / messaging feeling too high-friction or raising the barrier for participation). I have a way to let Claude call me now to tell me stuff when I have a bunch of instances out doing stuff and then leave to go home.

I'm trying to get that better integrated in my devloop because I think it makes managing >4 agents simultaneously much more feasible and natural for some people (I used to play Starcraft a lot so I'm used to the multitasking, but it still takes sustained willpower to be constantly "driving" or monitoring things, or to field questions), especially ones who have never served as TLs or people managers before. IMO it's a big performance roadblock for a lot of developers to be treat directing multiple agents simultaneously as some kind of high-stakes/high-cost thing. The kind of developer who would not say anything in a team meeting unless prompted or who thinks everything is stupid by default (because they are afraid of making decisions / being wrong even if only briefly) is both very common and reluctant to work this way, but also really probably needs it to be as productive as more skilled developers.

I type as fast as I talk so for majority of my LLM usage I don't need text to speech.

But I love the chatgpt voice interface e.g. on a long drive when I can use it to learn about random stuff (btw, turn advanced voice off for such usage).

Other part though is, hacker news vs regular population, majority of which would much much rather talk and listen than type and read.

Faster, and that's it. If you don't need precision (like with prompting LLMs) the speed gain is massive (*for most people)

I've been using ChatGTP by voice for things like cooking and house repair stuff. It's quite convenient for situations in which your hands are busy.

Other week I fixed a a water valve. After planning the thing with ChatGTP I brought the new valve. Then I described what I was seeing as I swapped the old valve for the new one to make sure everything was right. Really cool experience!

When I was still using OpenAI, I used it among other things to translate from English to Spanish while talking to Spanish-speaking people in person.

I understand a bit Spanish but I don’t speak Spanish yet, and they don’t speak English.

I speak English to the AI and end with “translate to Spanish, translation only”, and then the AI says the thing I was saying in Spanish (not perfect but good enough, and also it has a slightly weird accent that might be it using English or English influenced text to speech even when speaking Spanish sentences?).

Sometimes it's faster than swyping on a phone, but mostly I use it to learn about stuff and hash out ideas while driving.

This may sound strange and even callous, but I think it's appealing to people who are used to having employees. It's not about speech being a better interface, it's that thinking hard enough to sit down and compose a prompt is too much work if you're used to just yelling at someone.

Pity the managers with no one left to boss around besides the machines coming for their own jobs.

I was asked just yesterday if I could wire up [redacted] so that [redacted profession] could have a realtime voice interface while in the middle of performing [redacted]. My basic answer was yes, but it would be a bit slower than you want if something is going wrong, and it would probably be unethical for a whole lot of reasons.

Accessibility.

A lot of people are slow typists.

Much faster and better flow. Don't knock it til you've tried it.

I can talk faster than I can type.

it's very confusing. maaaybe if the stt is good and fast enough, speaking may be faster? english speakers can probably hit 150-180 wpm but seems like a hassle

It's easier, faster, and more natural to talk than to type for the vast, vast majority of people.

This trivial fact of life is observed every day by e.g.:

- students taking notes and finding it necessary to only jot down key facts so that they can keep up,

- stenographers who require special training and equipment to keep up verbatim with live speech in the courtroom,

- annoying colleagues who insist on "hopping on a quick call" or arranging big, wasteful, and disruptive meetings instead of just writing down their problem / sending a message or email,

- friends who insist on sending short voice messages in DMs instead of typing, because it's more "personal" that way (which to be fair it is, but not to the extent proclaimed).

It's not just the weights. It is the system prompt, harness, safety filters, etc. Those can affect performance of the same underlying model significantly.

Darling, we'll always have W_q, W_k, W_v, and W_o.

I think my main concern was productivity, but tell me more about this AI Girlfriend

Only images or videos as well?

Hànzì can use 30%-40% fewer tokens than English. So, yes, it probably thinks better in Chinese.

But why does it do so inconsistently, and sometimes even forgetting to swap back to English when it comes time to do 'normal' output? It also seems recent, as when I was using deepseek even a week ago this was very rare compared to what I was seeing yesterday. I had to start including a line asking it to stay to English because I can only speak/read English.

A chinese model which tells me it is Claude from Anthropic? Not really. Chinese HW yes, SW not.

What subscription?