himanshuvnm / Foundation-Model-Large-Language-Model-FM-LLM

This repository was commited under the action of executing important tasks on which modern Generative AI concepts are laid on. In particular, we focussed on three coding actions of Large Language Models. Extra and necessary details are given in the README.md file.

Foundation-Model-Large-Language-Model-FM-LLM-

This repository was commited under the action of executing important tasks on which modern Generative AI concepts are laid on. In particular, we focussed on three coding actions of Large Language Models which are given as follows:

1. We explore the dialogue summarization example through Generative AI on the AWS with instace-type ml-m5-2xlarge. This was successfully executed by incorporating summary of a dialogue with the pre-trained Large Language Model (LLM) FLAN-T5 from Hugging Face. Further, we employed Prompt engineering which is an important concept in foundation models for text generation. We used Zero-Shot Inference, One-Shot Inference and Few-Shot Inferences to conclude the dialogue summarization experiment.
2. In the second experiment, we explore an important concept of Fine Tuning on a Generative AI model and again we worked on the dialogue summarization experiment. It is again important to note that this particular experiment was conducted on the AWS with instace-type ml-m5-2xlarge. After recalling the dataset of our interest, which in this case is DialogSum Hugging Face dataset, we load the pre-trained FLAN-T5 model and then tokenize it. After testing the model with shot-inferencing, we Fine-Tuned the model and then we evaluated the validity of the trained LLM by the ROGUE metric. After that we have performed Fine Tuning, we executed Parameter Efficient Fine-Tuning (PEFT) which is a generic term that includes Low-Rank Adaptation (LoRA) and the experiment ic concluded by calculating the ROGUE metric again to check the validity of PEFT on the model.
3. Lastly, we study how to fine tune a FLAN-T5 model to generate less toxic content with Meta AI's hate speech reward model. After we have executed traditional commits, we perform the fine tuning to detoxify the summary by optimizing the Reinforcement Learning policy against the reward model by using Proximal Policy Optimization (PPO). Again, please do not forget that, we conducted all this on the AWS with instace-type ml-m5-2xlarge.

All these coding were made available during the course that I took on https://www.coursera.org/learn/generative-ai-with-llms at Coursera. The certificate of my active participation is already uploaded in this repository.