SLAMPAI / generative-models-for-highres-solar-images

A Comparative Study on Generative Models for High Resolution Extreme Ultraviolet Solar Images https://arxiv.org/abs/2304.07169

A Comparative Study on Generative Models for High Resolution Extreme Ultraviolet Solar Images

by Mehdi Cherti, Alexander Czernik, Stefan Kesselheim, Frederic Effenberger, Jenia Jitsev [arXiv]

• Short version of the paper accepted at ICML 2023 Astro workshop

solar.mp4

Introduction

In this repository, we provide the code for "A Comparative Study on Generative Models for High Resolution Extreme Ultraviolet Solar Images" (arXiv).

Obtaining solar (SDO) data

The dataset is available here: https://huggingface.co/datasets/slampai/solar-sdo. Download it and unzip using:

wget https://huggingface.co/datasets/slampai/solar-sdo/raw/main/image_folder_1024x1024_normalized_log_transform_193A_40K_with_lev1.5_corrections.zip
unzip image_folder_1024x1024_normalized_log_transform_193A_40K_with_lev1.5_corrections.zip

Downloading models

(Best) Diffusion model

wget https://huggingface.co/slampai/generative-models-for-highres-solar-images/resolve/main/diffusion/diffusion_1000t_lr0.0001_128ch_2bpr_horiz_flip/ema_0.9999_058000.pt --output-document=ema_0.9999_058000.pt

The full set of models is available at https://huggingface.co/slampai/generative-models-for-highres-solar-images/tree/main/models/diffusion.

(Best) ProjectedGAN model

wget https://huggingface.co/slampai/generative-models-for-highres-solar-images/resolve/main/models/projgan/00017-stylegan2-proj_baseline/network-snapshot.pkl --output-document=projgan_best.pkl

The full set of models is available at https://huggingface.co/slampai/generative-models-for-highres-solar-images/tree/main/models/projgan.

Sampling from models

For diffusion models, see colab.

For ProjectedGAN, see colab, also latent space exploration included.

Results

See results.ipynb.

Citation

If you find this work helpful, please cite our paper:

@article{cherti2023comparative,
  title={A Comparative Study on Generative Models for High Resolution Solar Observation Imaging},
  author={Cherti, Mehdi and Czernik, Alexander and Kesselheim, Stefan and Effenberger, Frederic and Jitsev, Jenia},
  journal={arXiv preprint arXiv:2304.07169},
  year={2023}
}

Ackknowledgements

• Diffusion code is based on OpenAI's ADM, thanks to the authors.

• ProjectedGAN code is based on https://github.com/autonomousvision/projected-gan, thanks to the authors.

• We would like to thank Ruggero Vasile for his contributions during the initial phases of the project with StyleGAN2 models, including data gathering, data pre-processing and preparation, as well as Yuri Shprits for support in the begining of the project.

• We would like to also thank Katja Schwarz for all her insights and experiment suggestions on our ProjectedGAN ablations.

• This work is supported by the Helmholtz Association Initiative and Networking Fund under the Helmholtz AI platform grant and the HAICORE@JSC partition. The authors gratefully acknowledge the Gauss Centre for Supercomputing e.V. (www.gauss-centre.eu) for funding this project by providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS Supercomputer JUWELS at the Jülich Supercomputing Centre (JSC). Partial support of DFG (SFB 1491) is acknowledged.