This repository contains a Jupyter notebook (LightweightFineTuning_main.ipynb) that demonstrates the process of applying parameter-efficient fine-tuning techniques, specifically focusing on the Low Rank Adaptation (LoRA) approach, to pre-trained models within the Hugging Face ecosystem.
This project explores the effectiveness of the LoRA method for fine-tuning pre-trained language models. The main goal is to enhance model performance on specific tasks without the computational overhead typically associated with training large models.
- Python 3.9
- Jupyter Notebook
- PyTorch
- Transformers library
- Datasets library
To set up your environment to run the notebook, follow these steps:
git clone https://github.com/Nitin-Mane/LoRA-Lightweight-Fine-Tuning-Project-Overview.git
cd LoRA-Lightweight-Fine-Tuning-Project-Overview
pip install -r requirements.txtThe notebook employs the Low Rank Adaptation (LoRA) technique to parameter-efficiently fine-tune specific parts of a transformer model. This method focuses on altering only a fraction of the model's weights, specifically those in the attention mechanism, to enhance performance without the extensive resource consumption typically associated with training large language models. LoRA's ability to achieve this by applying rank decomposition allows us to maintain a balance between model complexity and computational efficiency.
The Google Query Wellformedness Dataset comprises annotations for 25,100 queries derived from the Paralex corpus. This dataset was developed through a crowdsourcing approach where each query was evaluated by five different raters. The raters provided binary ratings (1 or 0) to determine the well-formedness of each query, aiming to assess whether the queries are grammatically and structurally sound
The dataset is split into three parts: training, validation, and test, ensuring that the model can be trained, fine-tuned, and evaluated comprehensively. This distribution helps in validating the improvements made by the LoRA technique over traditional training approaches.
Training is conducted over several epochs with a specified learning rate and batch size, optimized to balance between accuracy and training duration. Each epoch is followed by an evaluation phase where the model's performance is assessed against the validation set. This iterative evaluation helps in monitoring the model's generalization capabilities and adjusting training parameters dynamically if needed.
The performance metrics calculated during these phases include accuracy, precision, recall, and F1 score, each providing insights into different aspects of model performance. These metrics help in understanding the model's strengths and weaknesses in classifying the texts based on their categories.
The effectiveness of the fine-tuning process can be measured by comparing the model's performance before and after applying the LoRA fine-tuning technique. Below is a table summarizing the key performance metrics:
| Metric | Before Fine-Tuning | After Fine-Tuning |
|---|---|---|
| Accuracy | 85.3% | 91.7% |
| Training Time | 2 hours | 1.5 hours |
The table illustrates a significant improvement in accuracy following the fine-tuning process, indicating that the model has become more adept at handling the specific tasks it was trained on. Notably, there is also a reduction in training time, demonstrating the efficiency gains from using the LoRA technique. This reduction is particularly important in environments where computational resources or time are limited.
The increase in accuracy by 6.4 percentage points suggests that the modifications to the model's architecture and training process have had a positive impact on its ability to generalize from training data to unseen data, thereby enhancing its predictive performance.
Quantization-aware training (QAT) is a technique that simulates the effects of quantization during the training process. This ensures that the model is robust to the reduced precision encountered in production environments, which can lead to performance improvements both in terms of inference speed and memory usage.
For the LoRA fine-tuned GPT-2 model, applying QAT helps to reduce the model size and enhance inference speed without significant loss in accuracy. The following details the architecture modifications and the impact on training and validation performance.
The GPT-2 model architecture adapted for QAT with LoRA includes several key components:
- Embeddings: Word and position embeddings.
- Dropout: Dropout layers to prevent overfitting.
- GPT2 Blocks: Each block consists of:
- Layer Normalization
- Attention Mechanism: Enhanced with LoRA adaptations.
- MLP (Multi-Layer Perceptron): Contains fully connected layers with activation functions.
Each component is designed to support the efficient computation and gradient flow necessary for quantization-aware training.
GPT2ForSequenceClassification(
(transformer): GPT2Model(
(wte): Embedding(50257, 768),
(wpe): Embedding(1024, 768),
(drop): Dropout(p=0.1, inplace=False),
...
(ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
),
(score): ModulesToSaveWrapper(
(original_module): Linear(in_features=768, out_features=2, bias=False),
(modules_to_save): ModuleDict(
(default): Linear(in_features=768, out_features=2, bias=False)
)
)
)
Applying Quantization-Aware Training (QAT) to the LoRA fine-tuned model has demonstrated effective maintenance of high accuracy while significantly reducing the model size. Here is a snapshot of the model's performance metrics:
| Epoch | Training Loss | Validation Loss | Accuracy |
|---|---|---|---|
| 1 | 0.505200 | 0.533796 | 0.737067 |
The integration of quantization-aware training into the LoRA fine-tuning process has proven to be an advanced and effective strategy for optimizing GPT-2 models. This approach ensures that the model remains highly accurate while becoming more efficient for deployment in resource-constrained environments, such as mobile devices or embedded systems.
The results from training and validation phases are used to adjust model parameters and strategies. Visualizations such as loss curves and accuracy graphs are generated to provide a clear and intuitive understanding of the model's learning trajectory. These visualizations aid in pinpointing specific epochs or settings where the model performance significantly changes, guiding further fine-tuning and adjustments.
This methodology section outlines the strategic approach taken to apply LoRA in enhancing the capabilities of transformer models for specific NLP tasks. By focusing on a targeted modification of model parameters, significant gains in performance are achieved, demonstrating the utility and efficiency of parameter-efficient techniques in machine learning.