mvrl / GeoSynth

A PyTorch implementation of "GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis"

GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis

Srikumar Sastry*, Subash Khanal, Aayush Dhakal, Nathan Jacobs (*Corresponding Author)

This repository is the official implementation of GeoSynth [CVPRW, EarthVision, 2024]. GeoSynth is a suite of models for synthesizing satellite images with global style and image-driven layout control.

Models available in 🤗 HuggingFace diffusers:

GeoSynth:

GeoSynth-OSM:

GeoSynth-SAM:

GeoSynth-Canny:

All model ckpt files available here - Model Zoo

⏭️ Next

• Update Gradio demo
• Release Location-Aware GeoSynth Models to 🤗 HuggingFace
• Release PyTorch ckpt files for all models
• Release GeoSynth Models to 🤗 HuggingFace

🌏 Inference

Example inference using 🤗 HuggingFace pipeline:

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
import torch
from PIL import Image

img = Image.open("osm_tile_18_42048_101323.jpeg")

controlnet = ControlNetModel.from_pretrained("MVRL/GeoSynth-OSM")

pipe = StableDiffusionControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-2-1-base", controlnet=controlnet)
pipe = pipe.to("cuda:0")

# generate image
generator = torch.manual_seed(10345340)
image = pipe(
    "Satellite image features a city neighborhood",
    generator=generator,
    image=img,
).images[0]

image.save("generated_city.jpg")

📍 Geo-Awareness

Our model is able to synthesize based on high-level geography of a region:

🧑‍💻 Setup and Training

Look at train.md for details on setting up the environment and training models on your own data.

🐨 Model Zoo

Download GeoSynth models from the given links below:

Control	Location	Download Url
-	❌	Link
OSM	❌	Link
SAM	❌	Link
Canny	❌	Link
-	✅	Link
OSM	✅	Link
SAM	✅	Link
Canny	✅	Link

📑 Citation

@inproceedings{sastry2024geosynth,
  title={GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis},
  author={Sastry, Srikumar and Khanal, Subash and Dhakal, Aayush and Jacobs, Nathan},
  booktitle={IEEE/ISPRS Workshop: Large Scale Computer Vision for Remote Sensing (EARTHVISION),
  year={2024}
}

🔍 Additional Links

Check out our lab website for other interesting works on geospatial understanding and mapping:

• Multi-Modal Vision Research Lab (MVRL) - Link
• Related Works from MVRL - Link