protagolabs / NetMind-TF2

NetMind models development in tensorflow2

NetMind-Models-TF2 We develop the netmind platform in tensorflow2

install conda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

bash Miniconda3-latest-Linux-x86_64.sh

source ~/.bashrc

conda update --force conda

install the tensorflow and some other deps

• create netmind-tf2 env

conda create --name tf2 python=3.9

• load enviroment

conda activate tf2

• install the cuda and cudnn

conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0

• add cuda/cudnn to path, otherwise you will meet issue when running tensorflow as: Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory

nano ~/.bashrc

• add the following cuda/cudnn path to the end, change "your_home_dir".

export PATH=/your_home_dir/miniconda3/envs/tf2/lib:${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/your_home_dir/miniconda3/envs/tf2/lib:${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

• load again

source ~/.bashrc

conda activate tf2

• install tensorflow-gpu

pip install tensorflow-gpu==2.9.1

test your install

wget https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_5340.zip
unzip -q kagglecatsanddogs_5340.zip

python test_example.py

Enjoy different categories could be found in corresponding directory:

pretrain

finetune

etc..

Acknowledgement

Huggingface Transformers Language Modeling

Tensorflow

Thanks for my colleages Xiangpeng Wan and Yu Cheng for their kindly helps