Posted by swyx 9 hours ago
I had to modify the default ComfyUI workflows to use a GGUF quant (city96's ComfyUI-GGUF custom node, UnetLoaderGGUF in place of the stock loader) [0].
I use the model labeled Q5_K_M. There is Q8_0 available as well, which is 34GB and fits fine in 64GB unified memory if you keep resolution modest.
The main issue is speed, a ~9-second 480x864 clip at 20 steps takes me a bit over an hour. So this will be cool to try for the speed up alone.
There's a lot of great information and workflows available to follow on the r/StableDiffusion subreddit.
[0] https://huggingface.co/Abiray/MiniMax-H3-GGUF/tree/main/unet
Put Codex to work on deploying it now, hoping the speed can improve quite a lot :-) Thanks anyway
That's crazy, a RTX Pro 6000 does that in in 2-3 minutes (give or take, depending on your exact settings). LLMs don't make the difference between standalone GPU vs unified memory + CPU so obvious as diffusion models seems to do.
Seriously, very dumb model compared to what you can run locally, but holy moly is it FAST on one GPU, seriously impressive. Can't wait for those to be scaled up a bit to fit perfectly within 96GB VRAM, then they'll be competitive.
Anyway, good input!
> This misconstruction is very common, included in print publications spanning several centuries. It might be considered an alternative spelling, albeit still a mistaken usage.
Thanks though, I never actually knew so was helpful :)
> On the 128 GB M5 Max, clean end-to-end image+audio and embedded-video+audio renders completed in 74.58 and 76.99 seconds respectively, each with about a 40.1 GB peak physical footprint and zero swaps.
Looks like it uses 40GB? So your 96GB mac setup should work fine i guess (Model itself is 33B)
I noticed on a bar TV the other day that some of the Chromecast screensaver landscape photo credits were to Peter Norvig. They were really lovely pictures.