Prefect as an orchestrator
amackillop opened this issue · comments
I've been evaluating Prefect (on github) as an alternative to airflow for my team and was wondering how much effort it would take in order to automatically create prefect flows from a TFX Pipeline definition like how we can automatically create dags for airflow.
Since the prefect evaluation is going really well and I'm also interested in using TFX, I want to see if I can merge the two.
I'd be happy to help in this -> I think prefect would be a good addition to the orchestrator list!