chaliy / langsnapy

Snapshot your LLMs

Langsnapy - Tool to snapshot your LLMs, NLP Models and Chatbots

I am looking for better name

Idea behind this tool, is to maintain snapshots of what your model can answer.

How it works: You define a set of questions. This tool run this questions against your model and save the answers. Then you can run this questions again and compare the answers with the saved ones.

How to use

pip install langsnapy

Check examples folder.

How to test

poetry install -E runtime-deps --with dev
poetry run pytest .
poetry run ruff .