Render Airflow TaskGroups with Clusters
nathadfield opened this issue · comments
Airflow DAGs that utilise TaskGroups would be better represented if we were to make use of Clusters
The problem I have with this feature is that retrieving the information from airflow whether it belongs to a taskgroup or not is unsafe. We cannot retrieve it from the rest api directly as it currently only is a UI concept. I have to parse the task_id 😟.
But we can give it a try.
Yes, that's true and presumably gets trickier if there are multiple levels of TaskGroups? I know there is a desire to make TaskGroups more than just a UI construct so perhaps this will be easier in the future.
Exactly. I believe I have heard something similar, too. Once it becomes a not-only UI concept anymore, tackling this issue will become much easier.