Giters

NVIDIA / ChatRTX

A developer reference project for creating Retrieval Augmented Generation (RAG) chatbots on Windows using TensorRT-LLM

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Fine-tune WhereIsAI/UAE-Large-V1 embeddings for Models First ?

sergiosolorzano opened this issue · comments

commented

Hi,

The embeddings model used on default ChatRTX installation is WhereIsAI/UAE-Large-V1. To use the generated embeddings in an LLM model imported to ChatRTX, do I first need to fine tune a pre-trained LLM model with AnglE so that WhereIsAI/UAE-Large-V1 embeddings are compatible with an LLM? e.g.

angle = AnglE.from_pretrained('NousResearch/Llama-2-7b-hf', pretrained_lora_path='SeanLee97/angle-llama-7b-nli-v2')

Thank you !

commented

NO fine tuning is required