Giters

MeteoSwiss / ldcast

Latent diffusion for generative precipitation nowcasting

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About the index file of train/valid/test

bugsuse opened this issue · comments

commented

It's great and interesting work! Thanks for your sharing the nice code!

I have downloaded the ldcast-dataset.zip and unzipped it to the data directory, but an error was raised when I ran train_geoforecast.py:

FileNotFoundError: [Errno 2] No such file or directory: '/public/home/ldcast/scripts/../data/split_chunks.pkl.gz'

Could you provide this file? In addition, some files (e.g., cache/sampler_autoenc_train.pkl, cache/sampler_autoenc_valid.pkl, cache/sampler_autoenc_test.pkl) were not found. Could you provide these files?

I found that the work supports using the nwp data as extra input variables (it's very interesting). Do you have plans to release the nwp data?

Thanks a lot!

commented

Thanks for your report, the split_chunks.pkl.gz file was missing. It's added in the above commit.

The files in cache are, as the name implied, cache files. They should be created automatically if they don't exist. Creating them can take a while on the first training run but once created they allow the sample creation to be initialized faster. These files are quite large so I don't think it makes sense to distribute them.

commented

And yes, it's intended to allow the code to use NWP data as well but this functionality is not ready yet. Once we implement it properly we'll probably write another paper and release the data along with it.

commented

Thanks for your report, the split_chunks.pkl.gz file was missing. It's added in the above commit.

The files in cache are, as the name implied, cache files. They should be created automatically if they don't exist. Creating them can take a while on the first training run but once created they allow the sample creation to be initialized faster. These files are quite large so I don't think it makes sense to distribute them.

Thanks for the update! @jleinonen It works fine now!

commented

And yes, it's intended to allow the code to use NWP data as well but this functionality is not ready yet. Once we implement it properly we'll probably write another paper and release the data along with it.

Looking forward to the interesting work using NWP data!