I've been hosting my own since whisper.cpp appeared on the scene, thrown up on a server with a 3090ti. Even if there is better/faster stuff out today, it just keeps on working without any issues, the weights are tiny and it's faster than I could need. This is basically what you need to get this working today:
MODEL="/home/user/projects/ggml-org/whisper.cpp/models/ggml-large-v3-turbo.bin"
WHISPER_SERVER_BIN="/home/user/projects/ggml-org/whisper.cpp/build/bin/whisper-server"
"$WHISPER_SERVER_BIN" --model "$MODEL" --language en --host 127.0.0.1 --port 7812
I'm really hoping someone either contributes a good server example to the codebase (and is willing to help with issues) or use transcribe.cpp or the bindings to create a robust server in another language :) would be happy to link it from the main project directly as well
I did this investigation before creating transcribe.cpp it would have been much more convenient and save me literal months of work. Happy to share the repo and binaries produced as well, but it was mostly throw away work to profile how to ship accelerated ONNX in Handy.
It works very well, the integration is much easier than before, users have model choice. So happy that this exists!
Will definitely check this out since it sounds like it eases through the pain of dealing with these.
I assume this is going to make maintaining SubtitleEdit a lot easier from now on, too: https://github.com/SubtitleEdit/subtitleedit/.
Anyone know a good Windows app that's just a window that transcribes - and translates - whatever goes through your output device, and not the microphone like most apps do?
The M4 max has probably 10x the compute and memory bandwidth hahaha