Table of Contents
YiVal is an adaptable AI development framework, designed to provide a tailored experimentation experience. Whether you're a hands-on developer or leaning into automation, YiVal is equipped for both:
- Data Input: Choose between manual data input or let the framework handle auto-generation.
- Variations: Manually set parameter and prompt variations or utilize the automated capabilities for optimal settings.
- Evaluation: Engage with manual evaluators or leverage the built-in automated evaluators for efficient results.
On the horizon is the YiVal Agent, an ambitious addition aimed at autonomously driving the entire experimentation process. With its blend of manual and automated features, YiVal stands as a comprehensive solution for AI experimentation, ensuring flexibility and efficiency every step of the way.
YiVal is an innovative open-source project, rooted in the principles of the Yijing, focused on revolutionizing the end-to-end evaluation process of AI models. We aim to provide comprehensive insights into not only the individual components such as models and prompts but more importantly, the final output produced by their interaction.
Qian (The Creative, Heaven) 🌤️ (乾):
- Setup the framework for wrappers that can be used directly
in the production code.
- Set up the BaseWrapper
- Set up the StringWrapper
- Setup the config framework
- Setup the experiment main function
- Setup the evaluator framework to do evaluations
- One auto-evaluator
- Ground truth matching
- Human evaluator
- Interactive evaluator
- Reader framework that be able to process different data
- One reader from csv
- Output parser - Capture detailed information
- Documents
- Git setup
- Cotribution guide
- End2End Examples
- Release