cgfalcon / research-on-activtion-functions

A activation functions research focusing on how bio or non-bio activation functions impact model complexity

Instructions

This is the source code for the course project: SYDE552

Our project titled: Unlocking the Potential: Exploring the Impact of Biology-Inspired Activation Functions

Installing

All the required packages are listed in the requirements.txt

Source Code Structure

• Root
  • experiments: All the experiments been conducted
    • cifar10: experiments on CIFAR-10 dataset
    • minst: experiments on MINST dataset
    • sensitiveparams: experiments on sensitivity of parameters
  • src: Source code of our experiment framework
  • *.ipynb: notebook of initial exploration

Experiments

We ran a number of experiments throughout our project. All the experiments can be found under the experiments directory.