TaskIds that contain numbers or dashes are not supported

[Nathan Hadfield]

If you have a DAG which contains tasks where the id contains either numbers or dashes then, although the diagram Python file is created, it cannot be run to generate the diagram image.

e.g.

from diagrams import Diagram
from diagrams.generic.blank import Blank

with Diagram("airbyte", show=False):
    airbyte-job_sensor = Blank("airbyte_job_sensor")
    airbyte_trigger_async = Blank("airbyte_trigger_async")
    
    airbyte_trigger_async >> [airbyte_job_sensor]

➜  airflow-diagrams git:(master) ✗ python3 examples/airbyte_diagrams.py
  File "/Users/nathan/Projects/airflow-diagrams/examples/airbyte_diagrams.py", line 5
    airbyte-job_sensor = Blank("airbyte_job_sensor")
    ^
SyntaxError: cannot assign to operator

[Felix Uellendall]

Thank you @nathadfield for reporting this.

This is indeed an issue. I will try to fix it in the next patch. Or if you want feel free to open a PR :)

[Nathan Hadfield]

@feluelle No worries. I will definitely aim to take a look over the next couple of days.

[Felix Uellendall]

You can see here how the variables will be generated. We could simply do a node["task_id"]|replace("-", "_") and edge["task_id"]|replace("-", "_")

[Nathan Hadfield]

Digits in the task_id would still be a problem though, right?

[Felix Uellendall]

Yes we can remove/replace them with a regex expression easily as well.

[Nathan Hadfield]

FYI - I also noticed that DAGs which use TaskGroups also generate invalid diagram variables.

[Felix Uellendall]

Because of the .?

[Felix Uellendall]

If that's the case, the Pr should also fix this as it is hashing it and every hash prefixed with an underscore should be a valid python variable.

[Felix Uellendall]

@nathadfield could you please test if this is working now. You can install the latest release (candidate) by running pip install --pre airflow-diagrams. It should install 1.0.1rc1.

[Nathan Hadfield]

Because of the .?

Yep! I'll certainly take a look at the RC tomorrow.

[Nathan Hadfield]

@feluelle So, your changes do look like they fie the issues with task_ids although the node labelling is a bit of a mess.

This is perhaps for another version but might it be a good idea to put task groups into clusters?

[Felix Uellendall]

Great idea. Let's move this to a different issue, please. And I would also add that to a minor release instead of a patch release. So will continue today with releasing 1.0.1.