couler-proj / couler

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Home Page:https://couler-proj.github.io/couler/index.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

feat: make yaml dump configurable

dmerrick opened this issue · comments

Summary

Right now when you use Argo, an empty yaml file gets dumped after every exception (regardless of if it's related to couler/argo or not). At our company we find our logs and stacktraces littered with seemingly-random Argo templates.

For example:

...
...
  File "/path/hidden/but/you/get/it.py", line 63, in __init__
    raise ValueError(f"Cannot parse SMILES: {smiles}.")
ValueError: Cannot parse SMILES: "Cn1cnc2n(C)c(=O)n(C)c(=O)c12".
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: conformer-search-
spec:
  entrypoint: conformer-search
  templates:
    - name: conformer-search
      steps: {}

This unwanted behavior comes from right here: https://github.com/couler-proj/couler/blob/master/couler/argo.py#L283-L285

I propose we make this behavior configurable and turned off by default.

Use Cases

Anyone who uses couler as part of a bigger project probably doesn't want to see random empty templates all the time.


Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

@merlintang Do you recall if the existing code reference above is something required for a particular use case?

@terrytangyuan we can remove this part, this is from the error part of yaml generation.

@dmerrick Feel free to submit a PR for this.