NisaarAgharia / Indian-LawyerGPT

Fine-Tuning Falcon-7B, LLAMA 2 with QLoRA to create an advanced AI model with a profound understanding of the Indian legal context.

Indian Law AI: Fine-Tuning Falcon-7B & LLAMA 2 Language Models

Welcome to our exciting project where we are adapting two cutting-edge language models, Falcon-7B & LLAMA 2, to become proficient in Indian law.

Overview

Our adventure began with a modest 150 Q&As on Indian law. Now, we're charging ahead with an impressive dataset of 3300 instructions! This AI legal project combines:

• Falcon-7B & LLAMA 2: State-of-the-art language models, prepped and ready for legal training.
• PEFT & QLoRA: The dream duo for memory-efficient and high-performance model fine-tuning.
• Our Dataset: Comprehensive Indian law knowledge, spanning constitutional law, civil rights, and more!

Research Paper

Dataset Creation

Dive into our Dataset

Our dataset is designed with four key features: instruction, input, output, and prompt. Crafted to shape our models into AI law experts! Dataset on Hugging Face : https://huggingface.co/datasets/nisaar/Constitution_Of_India_Instruction_Set https://huggingface.co/datasets/nisaar/Articles_Constitution_3300_Instruction_Set https://huggingface.co/datasets/nisaar/LLAMA2_Legal_Dataset_4.4k_Instructions

Fine Tuning Process

Track the Progress

Get a front-row seat to the training progress with TensorBoard. Kickstart it, navigate to the provided localhost link, and witness the models learn: