HaokunChen245 / DFDG

[ACML 2022] Towards Data-Free Knowledge Distillation

DFDG (Data-Free Domain Generalization)

This repo contains code for our paper:

Towards Data-Free Domain Generalization
Ahmed Frikha*, Haokun Chen*, Denis Krompass, Volker Tresp
LMU, Siemens
ACML 2022

Installation

1. OPTIONAL (but recommended): Create a virtual environment using Python's builtin venv ...

   $ python -m venv .venv
   $ source .venv/bin/activate

   ... or Conda:

   $ conda create -n ENV_NAME python=X.Y
   $ conda activate ENV_NAME

2. Install Poetry and install dependencies:

   $ curl -sSL https://install.python-poetry.org | python3 -
   $ poetry install

Quickstart

Library

from dfdg.download.dataset_download import download_dataset
from dfdg.download.teacher_download import download_teacher
from dfdg.training.train import train
from dfdg.evaluation.evaluate import evaluate

# Download dataset.
download_dataset(DATASET_NAME, './data')

# Download pretrained teacher models.
download_teacher('./model')

# Train a student model for each domain using the pretrained teacher models.
student_dir = train(
    dataset=DATASET_NAME,
    dataset_dir='./data',
    lr_img_stage1=0.1,
    batch_size_stage1=128,
    batch_num_stage1=2,
    lambda_moment_stage1=1,
    iterations_img_stage1=200,
    slack_stage1=10,
    lr_img_stage2=0.1,
    batch_size_stage2=128,
    batch_num_stage2=2,
    lambda_moment_stage2=1,
    iterations_img_stage2=200,
    slack_stage2=10,
    lr_student=0.1,
    batch_size_student=256,
    iterations_student=5,
    model_dir='./models',
    seed=1,
)

# Evaluate the student models.
result = evaluate(
    dataset=DATASET_NAME,
    dataset_dir='./data',
    model_dir='./models',
    student_dir=student_dir,
    batch_size=1024,
)
print(result)

CLI

1. Download the dataset and the teacher models

   $ python -m dfdg dataset_download
   $ python -m dfdg teacher_download

2. Train a model:

   $ python -m dfdg train

3. Evaluate a model:

   $ python -m dfdg evaluate

Citation

If you use our code in your research or wish to refer to the results published in our work, please cite our work with the following BibTeX entry.

@article{frikha2021towards,
  title={Towards Data-Free Domain Generalization},
  author={Frikha, Ahmed and Chen, Haokun and Krompa{\ss}, Denis and Runkler, Thomas and Tresp, Volker},
  journal={arXiv preprint arXiv:2110.04545},
  year={2021}
}