bitisony / deep-learning-pytorch-huggingface

Getting Started with Deep Learning with PyTorch and Hugging Face

This repository contains instructions/examples/tutorials for getting started with Deep Learning Pytorch and Hugging Face libraries like transformers, datasets.

Training

• Getting started with Pytorch 2.0 and Hugging Face Transformers
• Fine-tune FLAN-T5 XL/XXL using DeepSpeed & Hugging Face Transformers
• Fine-tune FLAN-T5 for chat & dialogue summarization
• Getting started with Transformers and TPU using PyTorch
• Fine-tune Falcon 180B with DeepSpeed ZeRO, LoRA & Flash Attention
• Extended Guide: Instruction-tune Llama 2
• Quantize open LLMs using optimum and GPTQ

Requirements

Before we can start make sure you have met the following requirements

• AWS Account with quota
• AWS CLI installed
• AWS IAM user configured in CLI with permission to create and manage ec2 instances

Commands

echo 'export PATH="${HOME}/.local/bin:$PATH"' >> ${HOME}/.bashrc 

watch -n0.1 nvidia-smi