YiVal is an innovative open-source project, rooted in the principles of the Yijing, focused on revolutionizing the end-to-end evaluation process of AI models. We aim to provide comprehensive insights into not only the individual components such as models and prompts but more importantly, the final output produced by their interaction.
Qian (The Creative, Heaven) 🌤️ (乾):
- Setup the framework for wrappers that can be used directly in the production code.
- Set up the BaseWrapper
- Set up the StringWrapper
- Setup the config framework
- Setup the experiment main function
- Setup the evaluator framework to do evaluations
- One auto-evaluator
- Ground truth matching
- Human evaluator
- Interactive evaluator
- Reader framework that be able to process different data
- One reader from csv
- Output formatter
- Output to csv
- Basic Textual UI
- Output parser - Capture detailed information
- Documents
- Git setup
- Cotribution guid
- End2End Examples
- Release