Top
Best
New

Posted by cmrdporcupine 4 hours ago

Mercury 3: Diffusion at scale(www.inceptionlabs.ai)
196 points | 77 comments
simonw 3 hours ago|
Pelicans aren't bad, a little quirky: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

Those were via OpenRouter - here's another set I made by calling the Inception API directly: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

runj__ 2 hours ago||
Those are some far better .svgs than I got last Mercury 2.5. I tried with 3d modeling and got some cool results incredibly fast! Unfortunately no real improvement under iteration (wasn't expecting that much, it's so fricking fast).

But seeing 10 mercury instances iterating all at once (as a personal demo) on a single concept felt unreal.

vorticalbox 3 hours ago|||
I didn’t wonder why the bench marks were done in medium. The medium pelican is better than the high.

Maybe reasoning negatively effects the output after medium.

cmrdporcupine 3 hours ago||
How fast did they generate? That's the specific sell with this model

EDIT: I see, "3337ms"

How does that compare to your other artistic works?

simonw 1 hour ago||
Yeah I thought the speed was a bit disappointing - that's why re-ran against the API without going via OpenRourer, didn't seem to speed it up though.
simonw 3 hours ago||
I still don't understand why Google did effectively nothing with their Gemini Diffusion demo from last year.

It was really cool, and really fast, and just went quiet.

They did release a Diffusion Gemma a few months ago but I've not seen much of a splash from that one, either.

nolist_policy 2 hours ago||
It's not Googles fault. It's our fault, llama.cpp doesn't support it, vllm can't really offload to host ram and while huggingface transformers supports it, it only can do f16 quant for MoEs like Diffusion Gemma. Unusable for low spec machines where diffusion really could speed up things.

We're sleeping on a amazing model right in front of our eyes. With diffusion models you can properly ban phrases. You can decensor and deslopiffy the model output without touching a single weight or doing a single backwards pass.

LarsDu88 3 hours ago|||
Their preprint showed it worked well with small batch size, but most prod deployments right now have rather large batch sizes from serving multiple users
simonw 3 hours ago|||
That's a great answer, thanks. I'd missed that detail.
LarsDu88 3 hours ago||
Honestly, a lot of those issues could be remedied with more optimized kernels, which LLMs could definitely write quite well now. I wouldn't be surprised if some deployments of Gemini Flash switched to diffusion for cases where there is low utilization, since Google showed you don't need to retrain an autoregressive model to rework it for diffusion inference.
jjaskulowski 3 hours ago||
If that's true, that's the real answer, or more like anty-answer because you're saying basically: hey, nothing should stop them.
LarsDu88 3 hours ago||
Google's gonna Google...
gallerdude 2 hours ago|||
I wonder if it'd work better as a local model
famouswaffles 2 hours ago|||
Difussion Gemma performed a lot worse than its equivalent autoregressive counterpart (it was particularly bad because rather than a sort of constant-ish gap you might see between 2 model classes, the gap seemed to worsen the harder the task/benchmark) and the speed benefits got negated at scale. Not really surprising we haven't heard much.
clhodapp 2 hours ago|||
This talk serves as sort on update on where they are with it: https://youtu.be/r305-aQTaU0?si=j3IqG9BAT0uumPeA

At a high level: It doesn't benefit from batching to reduce memory bandwidth in the same way that a "standard" autoregressive model does, which ends up meaning that it's great at reducing latency for individual/low-concurrency users and delivers better outcomes on certain tasks but much more expensive per token at scale.

jstummbillig 3 hours ago|||
They probably did not get it to work well?
dflock 3 hours ago||
Good at research, terrible at product?
Escapado 1 hour ago||
Quick review: It's really fast and works really well in my initial tests. I used it to build a little Swift UI app that runs in the background and helps me OCR and translate text[1]. When I hit CMD + Shift + O it selects a portion of the screen (think of the screenshot tool), runs OCR on the portion and throws that at the google translate API and displays the result in a tooltip above / below the portion. The hotkey / key combination, translation provider and target language can be changed in the settings and there is a little tray icon from which I can access those and enable/disable the tool. I made it come up with a plan for this tool and had it implement the thing end to end in about 8 or 9 steps. It originally messed up the coordinate calculation but quickly corrected that when I pointed it out and other than that it one-shotted all the other steps. The code has tests and looks okay for what it is and it works really well so far. All of it took me around than 45 minutes of prompting, testing, correcting and approving changes. It used 26 million tokens (1.3M uncached) at around 160TPS. I used Oh My Pi as a harness.

[1]When I study chinese or japanese the PDFs I have often don't work with the macos preview selection tool and I don't want to go paste whatever I can select into another tool so I find this useful.

bigmadshoe 9 minutes ago||
Any idea why you only got 160TPS when the blog post claims 1000+TPS? That seems like the only draw to this model right now while capabilities are behind the frontier.
layla5alive 1 hour ago||
You should be able to skip the OCR step by grabbing the actual text from the windows via OS APIs, unless your PDFs are scanned and not encoding digital text.
Escapado 41 minutes ago||
Lots of them are scanned sadly! But I added an alt mode where a selection gets translated without the OCR step - though OCR is on the order of 100ms so I don’t mind it that much
bpavuk 1 hour ago||
love these guys. instead of spending time on something akin to those creepy "dots" or obnoxious @Claude in Slack, they ship actual improvements to speed. you can't imagine what kinds of UX a fast LLM can enable! and for coding I have always been a proponent of strong compile-time type systems and handing off only the actual boilerplate typing, all to maximize actual thinking and speed of high-precision prototyping.

I actually built a small prototype that integrates with JetBrains IDEs - it tracks your in-IDE activity (kinda like JetBrains' own "file-scale tab-completion" feature) but you can invoke Codex with all that context at any moment to let back-then GPT-5.4-nano finish the miniature task. say, you wrote a new use case and your goal is to expose it through a new endpoint, but it's kinda boilerplate-y and DRY is not your style either. then you open a needed file, press a shortcut, tell what you want, and you get it. JetBrains Air kinda absorbed that functionality, Codex tokens are hella expensive (don't get me started on API costs), and JetBrains is ready to front that cost through "Junie Lite" (which is currently Qwen 3.6 27B), so I decided to abandon that prototype. but I do miss the combination of speed and control over resulting code.

I may revisit this idea with Mercury 3, or ask JB to integrate it.

culanuchachamim 51 minutes ago|
I will argue that part of the push for this kind of new and novel methods are because the "dots" style of applications demand new technologies to work better.
nkoren 4 hours ago||
@volodia (Inception co-founder) has a comment here which is flagged/dead. That's weird; it shouldn't be. Can someone fix that?
dgellow 46 minutes ago|
Send an email to the mods, it’s the best way to notify them of issues like that
loufe 4 hours ago||
I'm very curious what the actual model size is and how the training process differed from a traditional LLM. Cool work regardless, incredible how catching up to a few months behind the cutting edge now seems like an achievable feat for new entrants, if still quite costly.
GodelNumbering 3 hours ago||
This looks noticeably more capable than previous gen. I purposely gave it a bit of a vague prompt to create an html renderer for UML diagrams, it produced impressive work. Then i asked a question, and in the third prompt I asked it to update the html which it did correctly.

Another thing that stood out, and this might just be a novelty thing, is that it picked different and less painful-to-eyes color schemes for each theme. Congrats to the team behind it.

Question (since their team has reps here and I have never used Inception via api): when you ask for a code change in your codebase (via coding agent), how much code does it need to read? How well does it handle tool calls? Since diffusion is a different way of doing things, I am curious about the code-needed for each incremental change in the codebase, say over a larger task.

dumberquestions 3 hours ago||
I'd be surprised if major labs aren't doing any diffusion experiments, and if they are, it would also be surprising if another lab is ahead of them in diffusion given just how much more compute they have at their disposal. My intuition could be way off though.
willy_k 3 hours ago|
Google more or less ordinated the category. They have now seemingly dropped it, ostensibly because it doesn’t scale economically like transformers do (via batching requests).

I think though, that if Inception can keep up these speeds and prices, assuming diffusion continues to improve (likely), that Flash and Luna could lose out for many use cases.

arendtio 45 minutes ago||
Having a chart with a linear scale on one axis and a log scale on the other axis is a really neat trick.

The resulting charts have a high probability of looking fantastic :-D

2001zhaozhao 1 hour ago|
I have liked diffusion models for quite a while, ever since i used the original Mercury to play AI Roguelite. I only have one question regarding Mercury 3: how tf did they get it to be so cheap?

If these benchmarks hold up it is a no-brainer against Haiku 5.5 and GPT-6 Luna, especially considering that Haiku 5.5 becomes 5x more expensive past 100k context, and this model goes up to 1M for the same price. (Note: the price increases 2x after a period post launch but is still way lower than Haiku is priced for >100k requests)

More comments...