Top
Best
New

Posted by simedw 10 hours ago

Show HN: I trained a 125M model to autocomplete piano on-device(simedw.com)
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15).

The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device.

The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.

442 points | 99 commentspage 3
Ohentis 4 hours ago|
It occurres to me that the relationship between pitches usually matters more than the pitches themselves. Perhaps the pitches could be encoded in a vector like what is typically done for position.
subhajeet2107 8 hours ago||
This is so amazing, can you improve the quality of generation at the cost of notes per seconds ? No one can play 108 notes/sec anyways, maybe you can train the model to do CoT for better quality
simedw 8 hours ago|
Yes, some kind of planning step is on my TODO list. Another thing I want to try is generating a few continuations in parallel, picking the one that looks best, and then continuing from there. Maybe the picking could be automatic.

I can probably squeeze out quite a bit more than 100 notes/sec as well. I haven’t spent much time optimizing inference yet.

dandaka 3 hours ago||
I would love to see a jamming partner. So I could play along him on the same piece.
dyzone 4 hours ago||
https://www.youtube.com/watch?v=wyo3JDMsWyM
gaya3bollineni 8 hours ago||
Running a 125M model on-device at that speed is impressive. How much did you have to optimize the model to get that performance on an iPhone?
simedw 7 hours ago||
The biggest speed improvement came from changing the note representation when I switched to compound note events: roughly 5× fewer autoregressive passes per note.

For the current model I’m using Core ML, which optimizes the kernels the first time you run it. I haven’t actually spent that much time tuning performance beyond that.

Naitik88 7 hours ago||
The answer about changing the note representation was interesting. Sometimes a change in how the problem is represented ends up giving a much bigger improvement than trying to optimize the model itself.
pil0u 8 hours ago||
Even after a few years deep into AI, I find your application absolutely magic. This is very inspiring, thank you for sharing.
leobg 8 hours ago||
I don’t have MIDI. How about whistling or playing the piano via microphone? Sounds easy. Another 6 month rabbit hole? :)
speedgoose 7 hours ago||
Talking about AI music with some live human MIDI inputs, Magenta Realtime 2 was released a few weeks ago and is pretty fun.

https://magenta.withgoogle.com/magenta-realtime-2

oleggromov 5 hours ago||
This is almost exactly what Jordan Rudess is doing now with some folks from MIT or Stanford right now. Google his interview with Rick Beato.
h2aichat 5 hours ago|
Congratulations! It is amazing!!! Can you do the same with a song and give different drums to see which one fits better?
More comments...