Top
Best
New

Posted by ChrisArchitect 1 day ago

Skyfall-GS – Synthesizing Immersive 3D Urban Scenes from Satellite Imagery(skyfall-gs.jayinnn.dev)
144 points | 36 comments
voidUpdate 1 day ago|
"explorable" and "immersive" is definitely a bold choice of words when you can't really get below the level of the buildings before the gaussian splatting is very obvious. Sure, it's impressive that you can get that detailed from a few satellite images, but I think that might be overselling it a bit
Animats 22 hours ago||
> you can't really get below the level of the buildings before the gaussian splatting is very obvious.

True. If you bring the viewpoint down to near street level and look horizontally, it's worse than traditional photogrammetry methods.

I've been looking for algorithms like this for representing distant regions in virtual worlds. Open Drone Map can do a good job, sometimes, but it really needs a cleanup pass.

makeitdouble 1 day ago|||
That's a matter of data, and this looks promising to me. If instead of satellite images they'd feed it drone shots they could probably get down to a level of detail that becomes actually immersive and would be way beyond the digital twins we currently have.
echelon 1 day ago||
We're early days. Models will soon interpolate all of that. Eventually in real time.

I wouldn't knock the research. The results look impressive to me.

Skyy93 1 day ago||
We probably won't. GS is a reconstructive method, so when data is unavailable, you can only perform poor interpolation. You would need additional generative, not reconstructive, models. However, this would open the door to unfaithful augmentation again.
echelon 1 day ago||
Different applications.

GIS won't want generative hallucinations.

Consumer mapping apps, social applications, and games (eg. flight sims) will want the maps to look as good as possible.

wkat4242 1 day ago||
GIS don't want half exploded buildings either. Nor would they care about photographic textures on the 3D models.
fsloth 1 day ago||
You’d be surprised what GIS - or at least GIS - adjacent customers want. If you think about any cute-but-useless map detail that comes to your mind there is likely a paying customer for it.
Qworg 1 day ago||
While the methodology wasn't published, Microsoft did something similar for Flight Simulator.

https://www.smithsonianmag.com/air-space-magazine/flight-box...

rkomorn 1 day ago|
If you want to dig more, I suspect Microsoft got a lot (if not all) of it from blackshark.ai . They're one of the companies whose logo shows up during game start.

I knew their name because, when I worked for an Airbus subsidiary, we talked with them about a solution to generate 3D environments for any/every airport.

They had some cool stuff but also some wonky stuff at the time (like highway overpasses actually being rendered as walls across the highway).

Stevvo 1 day ago||
blackshark.ai was used for generating building models with textured facades from footprints. Their thing for generating airports was actually not used in MSFS. The photogrammetry from satellite photos in MSFS 2024 was done by Maxar Technologies.
rkomorn 20 hours ago||
Ah, neat. How do you know this?

And also, was it any different for MSFS 2020?

Stevvo 11 hours ago||
My side-project is working on addons, so I spend a lot of time with Flight Simulator. 2020 didn't have the photogrammetry from satellite photos, only from aerial photos in major cities. The blackshark buildings are used wherever they don't have any photogrammetry available.
rkomorn 9 hours ago||
Neat (again :) ).
mtharrison 1 day ago||
Maybe dumb question but how do I just take a sat image and create the scene? The scripts in the repo are all about training which I assume requires you to have the 3d data too.

These sort of projects always look cool but I think the real "wow factor" would be a file upload where you can see the result on your image. I assume there are reasons why this isn't done.

pedalpete 1 day ago|
Do you mean converting your image into a 3D scene?

This is where we were heading with our 3D volumetric video company https://ayvri.com

We were working on blending 3D satellite imagery with your ground view (or low flying in the case of paragliders) photos and videos to create a 3D scene.

Our technology was acquired prior to us being able to fully realize the vision (and we moved on to another project).

r0x0r007 15 hours ago||
Seems like it could also be useful in planning combat(strategic) actions ad hoc, given limited resources, but I guess military already has some other tech for this...
thicknavyrain 1 day ago||
This is so cool. I used to work on urban heat island analysis and now work in natural catastrophe modelling, and in both cases knowing the average heights/volumes of buildings is a very handy thing to have but is surprisingly difficult information to retrieve. Even a coarse estimate available at annual resolution has some really awesome use cases, very excited to see this.
CobrastanJorji 1 day ago||
It looks good! I imagine a reasonable next step might be to do something about the cars, which are omnipresent in urban scenes but seem like they've been left a blurry mess in the examples.
anigbrowl 1 day ago||
Is there any reason that this couldn't integrate Street View data?
fsloth 19 hours ago|
For production scenarios data like that usually has a hefty license fee. Hence it’s interesting to find methods for scene generation from data that is cheaper to acquire.
nevster 22 hours ago||
I'm probably in a minority here. When I read Skyfall-GS, I immediately thought it was some new Apple IIGS game!
EagnaIonat 19 hours ago||
Pretty cool. Interesting that the shadows become 3 dimensional objects as well.
daemonologist 1 day ago|
Very cool; interesting how it turns all the trees into puffballs though. Some artifact of the pre-trained depth estimation or diffusion model maybe?
More comments...