argoproj / argo-workflows

Workflow Engine for Kubernetes

Home Page:https://argo-workflows.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Option to disable writing k8s events

tooptoop4 opened this issue · comments

Summary

Environment variable/config to stop writing k8s events

Use Cases

When would you use this? When i want to stop etcd filling up

These parts of the code are writing k8s events which end up being stored in etcd:

case wfv1.WorkflowRunning:
woc.eventRecorder.Event(woc.wf, apiv1.EventTypeNormal, "WorkflowRunning", "Workflow Running")
case wfv1.WorkflowSucceeded:
woc.eventRecorder.Event(woc.wf, apiv1.EventTypeNormal, "WorkflowSucceeded", "Workflow completed")
case wfv1.WorkflowFailed, wfv1.WorkflowError:
woc.eventRecorder.Event(woc.wf, apiv1.EventTypeWarning, "WorkflowFailed", message)

woc.recordNodePhaseChangeEvents(woc.orig.Status.Nodes, woc.wf.Status.Nodes)

With a lot of workflows being churned through these events add a lot of storage pressure to etcd. At the same time i have not found these events useful as i can always look at logs in the controller pod

# Whether or not to emit events on node completion. These can take a up a lot of space in
# k8s (typically etcd) resulting in errors when trying to create new events:
# "Unable to create audit event: etcdserver: mvcc: database space exceeded"
# This config item allows you to disable this.
# (since v2.9)
nodeEvents: |
enabled: true

  # Whether or not to emit events on node completion. These can take a up a lot of space in
  # k8s (typically etcd) resulting in errors when trying to create new events:
  # "Unable to create audit event: etcdserver: mvcc: database space exceeded"
  # This config item allows you to disable this.
  # (since v2.9)
  nodeEvents: |
    enabled: true

@terrytangyuan i already have that as false but it still sends events!! seems i am getting events at workflow level