predibase / json-mode-benchmark

JSON Mode Case Study and Benchmark

• conllpp.csv is a version of the CoNLL++ dataset that's been preprocessed to be more suitable for training an LLM on the NER task.
• benchmark.ipynb is a notebook illustrating how the LLM can be evaluated against test data.
• The benchmarks directory contains the result of running the LLM over the test dataset with each combination of fine-tuned adapter and JSON schema.