Pytorch implementation of Brain Language Model (BrainLM), aiming to achieve a general understanding of brain dynamics through self-supervised masked prediction.
Clone this repository locally:
git clone https://github.com/vandijklab/BrainLM.git
Create an Anaconda environment from the environment.yml file using:
conda env create --file environment.yml
conda activate brainlm
And check the installation of major packages (Pytorch, Pytorch GPU-enabled, huggingface) by running these lines in a terminal:
python -c "import torch; print(torch.randn(3, 5))"
python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('I love you'))"
Datasets are available on shared storage. Ask Syed or Antonio for more details.
To train a model on Yale HPC, see the example HPC job submission script in scripts/train_brainlm_mae.sh.
If the environment.yml file does not successfully recreate the environment for you, you can follow the below steps to install the major packages needed for this project:
- Create and activate an anaconda environment with Python version 3.8:
conda create -n brainlm python=3.8
conda activate brainlm
-
Install Pytorch:
conda install pytorch==1.12.0 torchvision==0.13.0 cudatoolkit=11.3 -c pytorch -
Install latest huggingface version:
pip install git+https://github.com/huggingface/transformers -
Install Huggingface datasets:
conda install -c huggingface datasets -
Install Pandas, Seaborn, and Matplotlib:
conda install pandas seaborn -
Install Weights & Biases:
conda install -c conda-forge wandb -
Install AnnData:
pip install anndata==0.8.0 -
Install UMAP:
pip install umap-learn -
Install Pytest:
conda install -c anaconda pytest