Giters

georgosgeorgos / few-shot-diffusion-models

Few-Shot Diffusion Models

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How to prepare data?

RogerQi opened this issue · comments

commented

It seems datasets are expected to be packed in .pkl file, but the instruction on how to prepare the .pkl files are missing. Alternatively, would it be possible to provide pre-processed pkl files?

Here are errors I got due to missing files.

Traceback (most recent call last):
  File "main.py", line 112, in <module>
    main()
  File "main.py", line 49, in main
    TrainLoop(
  File "/home/roger/reproduction/few-shot-diffusion-models/model/set_diffusion/train_util.py", line 389, in run_loop
    batch = next(self.data)
  File "/home/roger/reproduction/few-shot-diffusion-models/dataset/__init__.py", line 58, in create_loader
    dataset = select_dataset(args, split)
  File "/home/roger/reproduction/few-shot-diffusion-models/dataset/__init__.py", line 26, in select_dataset
    dataset = BaseSetsDataset(**kwargs)
  File "/home/roger/reproduction/few-shot-diffusion-models/dataset/base.py", line 56, in __init__
    self.images, self.labels, self.map_cls = self.get_data()
  File "/home/roger/reproduction/few-shot-diffusion-models/dataset/base.py", line 81, in get_data
    with open(path, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/gigi/ns_data/cifar100/train_cifar100.pkl'
commented

Resolved. There is an undocumented preprocessing script at https://github.com/georgosgeorgos/few-shot-diffusion-models/blob/main/dataset/util/preprocess_datasets.py

Hi, then how to run the preprocess_datasets.py with dataset cifar? I find that the type of dataset in it is h5py. How did you deal with it?