Giters

zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks

Home Page:https://privategpt.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Evaluation by RAGAs

psmukhopadhyay opened this issue · comments

commented

Hello

Is it possible to link PrivateGPT with RAGAs metrics (https://docs.ragas.io/en/stable/concepts/metrics/index.html#ragas-metrics) for evaluation? Can you please guide me how to achieve this?

Best

commented

Glad to heard about evaluation. You can start looking at llamaindex integration: https://docs.ragas.io/en/stable/howtos/integrations/llamaindex.html

If you check the doc, you will see that you can already use llamaindex on datasets to evaluate them. For LLM, you can inject LLMComponent and get the PGPT model. For the testing part of the retriever, you will have to externalise or give access to the retriever along with all the node_postprocessor.