Top
Best
New

Posted by amlib 5 hours ago

Blender 5.3 released with Gaussian Splat support(www.blender.org)
367 points | 52 comments
Retro_Dev 5 hours ago|
Every time I hear about Gaussian splatting, I think about triangle splatting[0] - GPUs were built to render triangles, so I wonder what games would be like if we supported generating and rendering them as much as we do Gaussian splats.

[0] https://trianglesplatting.github.io/

guitarlimeo 5 hours ago||
The Dreams game engine (only for PS4) does this and is able to achieve scenes that no other game engine can. Here's a technical talk about how they did it: https://www.youtube.com/watch?v=u9KNtnCZDMI

Actually sorry not sure if they use triangle splatting, but gaussian splats at least.

mrandish 4 hours ago|||
The Sony Dreams beta in 2018, was truly revolutionary and even today the world hasn't caught up with a lot of its breakthroughs. I'd been deeply involved in the design of high-end, pro digital media authoring tools for two decades and when I saw that beta it blew my fucking mind.

Media Molecule (LittleBigPlanet) spent over 8 years creating Dreams, which would never have happened without Sony's funding, but the tragedy was forcing what could have been a brilliant high-end authoring tool into a gamepad / lean-back interface. Even now, there are still innovative UX approaches in Dreams that could be adapted into high-end 2D, 3D, compositing and animation tools like Blender.

Boxxed 4 hours ago|||
Pretty sure they use SDFs, not triangle or gaussian splatting
debugnik 4 hours ago|||
If I recall their talk correctly, models are unions of SDFs, which get sampled into a point cloud, and each point gets splatted as another SDF primitive called a "fleck".
guitarlimeo 4 hours ago||||
Ah thanks for the correction. The splat terminology they also use confused me.
cma 2 hours ago|||
I think SDFs is the data underneath, but baked into SDF voxel tiles at runtime and not always evaluating SDF shapes. The SDF tiles still get splatted.

At one point it used triangles for the large splats, and still compute shaders for the small ones due to quad overdraw. I'm not sure if the final version stuck with that or went all compute, but quad complexity is also why nanite does compute shaders for small triangles and traditional rasterization for the larger ones.

mschuetz 2 hours ago||
In my experience it isn't really about quad overdraw/helper lanes, but rather about primitive assembly. On my RTX 4090, vertex shaders max out at about 20M tris/ms, while mesh shaders, which bypass primitive assembly, can reach 40M tris/ms. Maybe quad draw might be the next bottleneck after primitive assembly.

But it's a bit difficult to find the exact reason since profilers won't really tell you a concrete bottleneck when drawing vast amounts of tiny tris.

Ohentis 2 minutes ago|||
Has anyone doing this but with shared vertices
sorenjan 5 hours ago|||
They've made a follow up to that called MeshSplatting. https://meshsplatting.github.io/
lambda 4 hours ago|||
I think that Gaussian splatting is generally better, as the Gaussians are smoother and easier to optimize, if you can use them.

Things like triangle splatting are useful if you have to integrate into an engine that only support triangles. But with modern graphics pipelines, you can implement Gaussians quite efficiently, including newer techniques like Gaussian point splatting that can scale up seamlessly to enormous scenes:

* Video: https://www.youtube.com/watch?v=2AWv_nIfp-U

* Paper: https://jorisar.nl/gaussian_point_splatting/gaussian_point_s...

* Repo: https://github.com/JorisAR/gaussian-point-splatting

WithinReason 1 hour ago|||
Already done in 2023: https://mobile-nerf.github.io/

runs on a phone: https://storage.googleapis.com/jax3d-public/projects/mobilen...

xscott 2 hours ago|||
Yeah, there were other splats before Gaussian splats, so lots of cool possibilities depending on the style you're after. However, hard triangles have hard edges, and then you might be anti-aliasing those edges in some way. Gaussian splats can just be a triangle or two where the pixel shader feathers towards the edges.
ffsm8 3 hours ago|||
Isn't nanite in unreal engine exactly that?

To be clear, I'm not proficient in it's use, just watched a few videos about it's theory when it was initially announced

cyanmagenta 1 hour ago||
No, nanite is just having different levels of detail for objects, and which LOD gets selected is done automatically by the engine. So users can use ultra-high-def models (including 3d scans of real-world objects), and the engine automatically generates and uses lower polygon versions for when the object is far away or whatever.
ffsm8 1 hour ago||
Strange, when I looked into it that seemed to be just a desirable side effect with the rendering being done via the triangles - which was also supposedly the reason why you could directly use scanned models, despite them having millions of triangles in their model

But again, I've really only super superficial understanding of this, I just understood it like that back then

I learned about gaussian splattering mere months before nanite was introduced and their technical talks and articles always sounded exactly like what I just learned about the splattering, that's why I got that impression

weryberl5 4 hours ago||
Honestly there should be many more types of splatting imo...
zokier 2 hours ago|||
Beta splatting seems pretty promising. It addresses the fuzziness aspect of gaussians among other things.

https://rongliu-leo.github.io/universal-beta-splatting/

https://linus-franke.com/hibs/

cubefox 3 hours ago|||
There are various papers on variants of Gaussian splatting (keyword "differential rendering") which replace the Gaussians with some other explicit geometric representation that can also be rendered in real time. A triangle soup is just one example. I recall one which uses some sort of tetrahedral foam instead.
mturk 5 hours ago||
Bundle support in geometry nodes is huge for complex pipelines and workflows, particularly when combined with lists and closures. I have been running on the alpha builds to get access to them in my workflows.

It took me a while to properly understand it, but it's genuinely great. It maps so much more of what I expect from traditional coding, scripting etc into the visual paradigm and makes an awful lot easier to use.

Also, should out to databpy and nodebpy, both of which are indispensable and make using nodes and blender so much nicer:

https://github.com/BradyAJohnston/databpy https://github.com/BradyAJohnston/nodebpy

unshavedyak 4 hours ago||
If you enjoy or get value out of Blender please consider donating: https://fund.blender.org/

They have easy subscription options, even a monthly coffee adds up! Thanks!

guidopallemans 3 hours ago||
Tried the beta on my M4 a couple of weeks ago on a scene of 4M splats (exported from spirula studio, great tool) , and found that it's immediately very mature, fast, and well-integrated. I could not find anyinconstencies with the rest of the interface. Congrats to the team!
load-bearing 5 hours ago||
Interesting discussion from start of this year of one of the first popular music videos made with this technology. Some decent explanations in there.

https://news.ycombinator.com/item?id=46670024

PyWoody 4 hours ago|
Wow, that video made me instantly nauseous.
Groxx 4 hours ago||
If you'd like a somewhat more sedate example: https://youtu.be/J1-yToxKOzI

Their LazyPosy channel has a video about how they made this one, difficulties, etc if you're interested in that too. More practical than technical, though it wasn't done with Blender.

utopiah 29 minutes ago||
Even works in XR, tried on Frame just yesterday https://video.benetou.fr/w/1z3RE28EGWB7ZeeAgdCaEY and don't trust the low FPS capture, was quite smooth.
dvh 4 hours ago||
You show something to your boss, he pats you on the shoulder and says "Perfect! Just move chairs 2 feet left and ship it". How does one do it with gaussian splats?
pfg_ 26 minutes ago||
You select the points involved in the chair and move them and then you fix the holes in the floor. Gaussian splats are just 3D point clouds with a bunch of extra values per point.
andybak 26 minutes ago|||
With about the same amount of ease as if you'd used photogrammetry - which is very widely used in game pipelines
derac 3 hours ago||
I have seen techniques for smart selecting objects and moving them around in the scene.
gregsadetsky 4 hours ago||
Does anyone have recommendations for video / 4d splats? Are there softwares / techniques to check out? How to record, render, etc.

Thanks!

originalvichy 3 hours ago||
I am a fan of this channel: https://www.youtube.com/@RadianceFields

He is very frequently posting news about the splat world as well as quite many tests and demos with different software or hardware. I would expect him to cover Blender specific stuff soon, but it's a great source already for existing stuff.

adventured 32 minutes ago||
Minimax H3 can do orbit videos + splats very well.
devy 5 hours ago||
Apparently, while Gaussian splatting is not the core technique for all world models, it has become a dominant representation and rendering backend for spatial and reconstructive world models.
jfrei 4 hours ago|
Pretty cool that Gaussian Splatting is included in Blender. It always amazes me how many (sometimes pretty hidden) features Blender provides while still being rather a lean piece of software. I believe though that Gaussian Splatting is not ideal for getting well-structured mesh data with limited vertices for typical video game usage. I'm wondering where this is going in the future.
More comments...