NaNPusher / YiVal

YiVal is a dynamic AI experimentation framework, offering a blend of manual and automated tools for data input, parameter variations, and evaluation. With the upcoming YiVal Agent, it promises to autonomously streamline the AI development process, catering to both hands-on developers and automation enthusiasts.

YiVal

Table of Contents

• About

---

About

YiVal is an adaptable AI development framework, designed to provide a tailored experimentation experience. Whether you're a hands-on developer or leaning into automation, YiVal is equipped for both:

• Data Input: Choose between manual data input or let the framework handle auto-generation.
• Variations: Manually set parameter and prompt variations or utilize the automated capabilities for optimal settings.
• Evaluation: Engage with manual evaluators or leverage the built-in automated evaluators for efficient results.

On the horizon is the YiVal Agent, an ambitious addition aimed at autonomously driving the entire experimentation process. With its blend of manual and automated features, YiVal stands as a comprehensive solution for AI experimentation, ensuring flexibility and efficiency every step of the way.

Screenshots

Best Parameter Combination

Data Analysis

Test Cases Side by Side

Roadmap

Qian (The Creative, Heaven) 🌤️ (乾):

• Setup the framework for wrappers that can be used directly in the production code.
  • Set up the BaseWrapper
  • Set up the StringWrapper
• Setup the config framework
• Setup the experiment main function
• Setup the evaluator framework to do evaluations
  • One auto-evaluator
  • Ground truth matching
  • Human evaluator
• Interactive evaluator
• Reader framework that be able to process different data
  • One reader from csv
• Output parser - Capture detailed information
• Documents
• Git setup
• Cotribution guide
• End2End Examples
• Release