Robust Conditional 3D Shape Generation from Casual Captures

Posted by lastdong 8 hours ago

Robust Conditional 3D Shape Generation from Casual Captures(facebookresearch.github.io)

43 points | 5 comments

fxtentacle 11 minutes ago|

This turns point clouds into meshes.

That means it doesn’t need depth. Depth is helpful for getting good point locations, but SLAM on multiple frames should also work.

I’m guessing that they are researching this for AR or robot navigation. Otherwise, the focus on accurately dividing the scene into objects wouldn’t make sense for me.

nico 4 hours ago|

Does this need depth data capture as well? The “casual captures” makes it seem like it only needs images, but apparently they are using depth data as well

Also, can it run on Apple silicon?

KaiserPro 1 hour ago||

Nope, only needs depth for ground truth.

its designed to be run on top of a SLAM system that outputs a sparse point cloud.

on page 4 on the top right you can see how the point cloud is used to then feed into the object generator: https://cdn.jsdelivr.net/gh/facebookresearch/ShapeR@main/res...

lastdong 2 hours ago||

I think it does use depth data from parameters in docs: python infer_shape.py --input_pkl <sample.pkl> (possibly achievable using software like MapAnything). I believe CUDA only.

efskap 32 minutes ago||

Yeah they confirm that at the bottom of the linked page

> Furthermore, by leveraging tools like MapAnything to generate metric points, ShapeR can even produce metric 3D shapes from monocular images without retraining.