sash-a / CoDeepNEAT

An implementation of CoDeepNEAT using pytorch with extensions

CoDeepNEAT

An implementation of implementation of CoDeepNEAT, originally created by Risto Miikkulainen et al. with our own extensions. Implementation details were taken from their 2017 and 2019 paper.

Setup

Requires conda

conda create -n cdn --file requirements.txt
conda activate cdn
pip install tarjan wandb  # these are not available from conda

Entry points

Directory: src/main/
ft.py Fully trains a run from evo.py
evo.py Does an evolutionary run
batch_run.py Running many different configurations all the way from evolution to fully training. (See note below)

Config

All config options are in src/configuration/configuration.py
Example configs are in src/configuration/configs directory

How to run

python src/main/evo.py -g 1 -c base

Extensions

Extensions are detailed in the paper linked above

Paper

If you use this code, please cite our paper:

@INPROCEEDINGS{9308151,
  author={S. {Acton} and S. {Abramowitz} and L. {Toledo} and G. {Nitschke}},
  booktitle={2020 IEEE Symposium Series on Computational Intelligence (SSCI)}, 
  title={Efficiently Coevolving Deep Neural Networks and Data Augmentations}, 
  year={2020},
  volume={},
  number={},
  pages={2543-2550},
  doi={10.1109/SSCI47803.2020.9308151}}

Results

For detailed results see:
convergence
evolution

The accuracies obtained on CIFAR-10

The best data augmentations found

The best genotype found. Using config configuration/configs/experiments/mms_da_pop_25e.json and a feature multiplier of 5

And its corresponding phenotype

Note about batch runs

This system was developed for rapid tuning of CDN's own hyperparameters on a cluster with a limited number of GPUs. It should not be used for normal training as it was created for our very specific case. Rather do a single run on evo.py and then fully train it with ft.py.