argoproj / argo-workflows

Pre-requisites

I have double-checked my configuration
I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
I have searched existing issues and could not find a match for this bug
I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?

We need to add some tolerations and node selector in workflow definition

Pods are getting rendered with nodeSelector but added tolerations are not being rendered into pod definition

Version

3.5.5

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

  apiVersion: argoproj.io/v1alpha1
  kind: Workflow
  metadata:
    generateName: x-y-
  spec:
    workflowTemplateRef:
      name: dummy-template
      namespace: argo-workflows
    entrypoint: aa-bb
    nodeSelector:
      stack: workflows
    tolerations:
    - key: stack
      operator: Equal
      value: workflows
      effect: NoSchedule
    - key: "kubernetes.azure.com/scalesetpriority"
      operator: "Equal"
      value: "spot"
      effect: "NoSchedule"`

Logs from the workflow controller

No Node found, as nodes have tolerations

Logs from in your workflow's wait container

No Node found, as nodes have tolerations

Do you have a complete example for this.

argo-workflows/workflow/controller/workflowpod.go

Line 788 in 2f15709

// Set tolerations (if specified)

appears to do the right thing, and the same thing as for nodeSelector, so I'd like to be sure you haven't got tolerations in your workflowTemplate you're running. They don't get merged.

@Joibel Yes, i confirm there are no tolerations/node selector in workflowTemplate, but still its not rendering it in pods.

apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
name: sample-sensor
namespace: argo-workflows
spec:
template:
serviceAccountName: sample-workflow-sa
dependencies:
- name: sample-jobs
eventSourceName: azure-queue-storage
eventName: processing-preview-jobs
transform:
jq: .body|=(@base64d |@base64d | fromjson)
filters:
data:
- path: "body.data.filePath"
type: string
value:
- '.-da-.-(full|snapshot|base)_[0-9]+.mcap'
triggers:
- template:
name: sample-workflow
k8s:
operation: create
source:
resource:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: sample-preview-
spec:
entrypoint: sample-entrypoint
nodeSelector:
stack: workflows
tolerations:
- key: stack
operator: Equal
value: workflows
effect: NoSchedule
- key: "kubernetes.azure.com/scalesetpriority"
operator: "Equal"
value: "spot"
effect: "NoSchedule"
synchronization:
semaphore:
configMapKeyRef:
name: workflow-config
key: SAMPLE_JOBS
arguments:
parameters:
- name: filePath
# this is the value that should be overridden
value: empty
workflowTemplateRef:
name: sample-workflow-template
namespace: argo-workflows
parameters:
- src:
dependencyName: sample-jobs
dataKey: body.data.filePath
dest: spec.arguments.parameters.0.value

I still need a complete example.

I don't think the trigger mechanism (events) matters here, but I can't run it as is because the workflow template referred to isn't provided.