Top
Best
New

Posted by c1b 1 day ago

Show HN: Watch a neural net learn to play Snake(ppo.gradexp.xyz)
In browser PPO training demo, made possible by tinygrad: TinyJit -> WebGPU kernels.

Requires WebGPU.

118 points | 30 comments
anthonycoslett 16 minutes ago|
Mesmerizing - could be its own digital art showcase XD Love what you've done here, friend. Looking forward to what you do next. <3
ldoughty 8 hours ago||
Really cool! But right as it was nearing 4,000, it seems to have corrupted itself and no longer got any scores above 0. Not sure if that's a code bug or a neural net issue.

avg500 -4.6 last 500 episodes

peak 3959.3 best window

roll/s 20.68 20-step avg

progress 4388 562749 episodes

c1b 6 hours ago||
Yes it just collapses eventually — never stabilizes. The training process is flawed, I suspect it has to do with the fact that some weights blow up over time, you can see in “weights” tab.

But at around 4K avg score you should see it solve the env almost every time.

Just a demo :) optimized for speed over stability.

Reward structure: Step: -1 Dot: +100 Win: +1000 so ~4k is max theoretical score on 6x6.

ticulatedspline 2 hours ago||
maybe because it doesn't understand "done"? perfect play is impossible, random variance will cause scores to drop even if the model plays well and "wins". feels like it would get stuck in a loop trying to improve what can't be improved.
fc417fc802 2 hours ago|||
The optimizer doesn't need to understand anything it's just an iterated mathematical construct. The author simply didn't bother to implement the necessary details to ensure numerical stability.

Alternatively it might be a problem with the scoring model in the end game.

jesuo 1 hour ago|||
feels like it would get stuck in a loop trying to improve what can't be improved.

That is the point, there is nothing on an intention that we cannot improve, the goal here is no more than 1 unique iteration of the same path

r3trohack3r 7 hours ago|||
I think I noticed it reach “end game.” The snake reaches a point where, if it gets any longer, it is out of squares and hits its own tail. So it finds the route through the squares that it can infinitely loop, never eats the ball, and score starts dropping and goes negative.
simedw 9 hours ago||
Cool project!

I noticed that if you go from training to watch and then back, the training temporarily drop significantly in score.

bguberfain 7 hours ago|
It seems to be something related the moving average calculation. So it is just a glitch on the chart.
joshka 4 hours ago||
A previous similar idea running as a Ratatui based TUI: https://github.com/bones-ai/rust-snake-ai-ratatui
snats 6 hours ago||
did a pretty similar thing last month for the text rendering library last month.

trained and made a viz for the model and then made it displace text.

should probably do a proper write-up:https://x.com/i/status/2038367016969724259

ziofill 4 hours ago||
I noticed snake gets penalized for not getting to the apple early, is that what you really want? Snake is about how long it gets not about the balance between length and wall clock time
foo12bar 4 hours ago|
But if not the snake could go into an infinite loop, never growing, never eating.
jesuo 4 hours ago||
Poorly programmed, it doesn't learn from its mistakes, the games get stuck in a loop because the snake doesn't capture a piece but the piece remains and there's a gap, constantly moving the snake along the same path with negative scores in an infinite loop leaving an unaltered yin and yang ;) there's a repetitive pattern in these infinite games between the position of the gap and the piece
spectre9 3 hours ago|
Did you let it train? This doesn’t happen for me
jesuo 3 hours ago||
Yes, thousands of games, you can see how it happens in the displayed game matrix, there comes a point when they all enter those loops https://ibb.co/bM4RPzPb
spectre9 2 hours ago||
Makes sense, author mentioned training collapses eventually
beardsciences 8 hours ago||
My average eventually made it to about 3900, and then stagnated between 3600-3900. I'm curious if this is universal behavior or not. I'm up to about 5k steps.
mavdol04 5 hours ago|
That's cool, i did exactly the same few years ago
More comments...