Public repository for neuron embedding paper.
This repository is the official implementation of Representing Individual Neurons for Neuroevolution.
This code was tested on Python 3.8.
To install requirements:
pip install -r requirements.txt
Visdom should be running in order to view the plots.
To train the models in the paper, run this command:
python run_train_test.py -m <modeltype> -e <epochs> -d <embedding dimension>
python run_train_test.py -c <configname>
configname
is the name of the configuration found in configs/settings.conf
. All training settings should be changed there.
Example:
python run_train_test.py -c Emb
<modeltype>
can be one of FC (fully connected, direct encoding), Conv (convolutional, direct encoding), Emb (fully connected, neuron embedding), EmbConv (convolutional, neuron embedding).
After every epoch, training model will be saved as epoch<epochnum>.pth
in the models
folder.
Results from evaluation are displayed in Visdom. Ensure Visdom is open before running.
To evaluate a trained model, run:
python run_train_test.py -m <modeltype> -d <embedding dimension> -l <model to load>
Example:
python run_train_test.py -m Emb -d 64 -l models/trained_to_ref_1.pth
To train a model to match a reference, run:
python run_train_to_ref.py -r <target fully connected model> -s <path to save model> -c <fully connected config setting> -e <embedded model config>
Example:
python run_train_to_ref.py -r models/fc.pth -s models/tmp.pth -c FC -e Emb_ref
To run the crossover experiment, run:
python run_interpolation.py
Settings for this experiment are in crossover.conf
.
To run the crossover experiment, run:
python run_crossover.py -l <layer number>
Example:
python run_crossover.py -l 1
Settings for this experiment are in crossover.conf
.
Pretrained models are included in the models folder.
fc.pth - first fully connected model, trained from random initialization
fc2.pth - second fully connected model with same settings, different random seed
mnist_conv_embedding_32dim.pth - 32 dimensional convolutional embedding model
mnist_conv_embedding_64dim.pth - 64 dimensional convolutional embedding model
mnist_neuron_embedding_64dim.pth - fully connected neuron embedding model, trained from random initialization
trained_to_ref_1.pth - fully connected neuron embedding model, trained to match weights of fc.pth
trained_to_ref_2.pth - fully connected neuron embedding model, trained to match weights of fc2.pth