Learn how to use logit bias with OpenAI models to create highly-powerful classifiers in minutes.
This notebook will walk you through using OpenAI's logit bias functionality to force the model to output only the classifiers we want, drastically improving LLM performance on classification tasks. With this approach, we can (in minutes) create classifiers that outperform custom-trained classifiers simply by using off-the-shelf LLM APIs.
This notebook shows a) how to make a standard 'true'/'false' classifier, and b) how to make a custom classifier for more complex tasks.
Use it in Google Colab with this link, or download the .ipynb file in the repo.
Beyond demonstrating how these systems are built, this notebook also provides a benchmarking system to test the performance of multiple LLMs against your target tasks.
The system will output easy-to-read performance tables, like this:
Below these tables, you'll be able to see the latency of each model. With both of these, you should be able to choose the best model for your use-case.
After you've created a classifier you're happy with, you can use the templates in the bottom two cells to get your classifiers ready for production use.
This project is MIT licensed.
Matt Shumer - @mattshumer_ Project Link: https://github.com/mshumer/openai-logit-bias-classification-walkthrough