ryardley / korrect

Framework for hallucination detection and correction in LLMs

Korrect

Korrect is a light-weight hallucination testing framework for LLMs.

To get started:

from korrect import Korrect

k = Korrect(ui=True)

then navigate to http://localhost:8501/ and you'll find the UI:

right now, korrect only supports openai models.

Korrect also supports experiments in code-only mode:

credentials = {
    "OPENAI_API_KEY": "*",
    "SERPER_API_KEY": "*"
}

client = Korrect(ui=False, credentials=credentials)

parameters = {
    "prompt": "who won the Nobel Peace Prize in 2020?",
    "models": [
    {"model_type": "OpenAI Chat",
    "model_name": "gpt-3.5-turbo"},
    {"model_type": "OpenAI Chat",
    "model_name": "gpt-4"}
]}

experiment = client.create_experiment(parameters=parameters)

results = experiment.run()

Development

For development, please run:

python3 -m venv myenv
source myenv/bin/activate
pip3 install -e .

Roadmap

• add Huggingface model support
• fix async await calls
• dockerize
• env config
• fix yaml file directory error
• fix import error ModuleNotFound

Contributions

To contribute, please see our contribution guide.

Community

Join our discord community here