allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

Home Page:https://clear.ml/docs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Human-in-the-loop pipeline step

MightyGoldenJA opened this issue · comments

Proposal Summary

Some pipeline steps, especially after quality control should be able to pause the execution of the pipeline and prompting the user for confirmation before proceeding to the next steps, this is common feature in most CI pipeline tools. The step should execute pre-hook code to prepare QC report or even be integrated with ClearML report so the human user can directly see data to make his decision. The UI should also have a Continue and Abort buttons.

Motivation

This would allow for more flexible validation of pipeline-shipping pipeline than programmatic quality criterion.