Inconsistent behavior with combining binding contexts

Question

Inconsistent behavior with combining binding contexts

knkarthik opened this issue 3 years ago · comments

According to https://github.com/flant/shell-operator/blob/master/HOOKS.md#shell-operator-lifecycle, If there is a sequence of hook executions in a queue, then hook is executed once with an array of binding contexts. However, I've noticed that this is not consistent - sometimes it is combined and tasks are dropped from the queue, other times all tasks are executed. I'm attaching hooks and logs below.

Expected behavior (what you expected to happen):
Since I want to run cleanup tasks based each resource that trigged the Deleted event, when I run kubectl delete applications.argoproj.io aut-410 aut-369 I expected it to always run twice - one each for aut-410 and aut-369 resource.

Actual behavior (what actually happened):
Sometimes it runs for only one resource and sometimes two.

Anything else we should know?:

I inspected the json file pointed to by BINDING_CONTEXT_PATH and confirmed that it has both the objects.

Hook script

#!/usr/bin/env ruby
# frozen_string_literal: true
require 'json'
config = {
  "configVersion": 'v1',
  "kubernetes": [
    {
      "name": 'OnCreateDeleteArgoApp',
      "apiVersion": 'argoproj.io/v1alpha1',
      "kind": 'Application',
      "executeHookOnEvent": %w[
        Deleted
      ],
      "labelSelector": {
        "matchLabels": {
          "app": "qa"
        }
      },
      "allowFailure": true
    }
  ]
}
if ARGV.size >= 1 && ARGV.first == '--config'
  puts config.to_json
else
  contexts = JSON.parse(File.read(ENV['BINDING_CONTEXT_PATH']))
  puts "---#{contexts.length}"
  contexts.each do |content|
    binding_name = content['binding']
    resource_event = content['watchEvent']
    resource_name = content.dig('object', 'metadata', 'name')
    resource_namespace = content.dig('object', 'spec', 'destination', 'namespace')
    if binding_name == 'OnCreateDeleteArgoApp'
      if resource_event == 'Deleted'
        puts "---#{resource_name} was deleted from #{resource_namespace}---"
      end
    end
  end
end

Logs when contexts are combined

fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-410: send KubeEvent","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"27619f3d-787c-4279-8758-b18ee71ef847","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-410'","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"27619f3d-787c-4279-8758-b18ee71ef847","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"1abf5006-ff92-45e7-9531-03e5019f3e53","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-369: send KubeEvent","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"0e7413ad-035d-41b2-826e-8ba0eb223575","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-369'","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"0e7413ad-035d-41b2-826e-8ba0eb223575","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"a92bf736-6276-400c-9100-2b2696036476","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: get task HookRun","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: tasks after wait [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=1abf5006-f], [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=a92bf736-6]","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"info","msg":"Binding contexts from 2 tasks are combined to 2 contexts. 1 tasks are dropped from queue 'main'","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Execute hook","queue":"main","task":"HookRun","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"debug","msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T18:57:56Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"-----2","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T18:57:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"---aut-410 was deleted from aut-410---","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T18:57:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Usage: \u0026{Sys:6.9ms User:106.952ms MaxRss:39592}","queue":"main","task":"HookRun","time":"2021-09-29T18:57:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Hook executed successfully","queue":"main","task":"HookRun","time":"2021-09-29T18:57:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: tasks after handle ","time":"2021-09-29T18:57:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: wait for task, delay 0","time":"2021-09-29T18:57:57Z"}

Logs when contexts are not combined

fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-410: send KubeEvent","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"29d0d561-c79a-4086-a42b-3c97df56ad87","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-410'","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"29d0d561-c79a-4086-a42b-3c97df56ad87","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"e56af739-fa33-4403-8503-5ba36a6e08c1","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: get task HookRun","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: tasks after wait [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=e56af739-f]","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Execute hook","queue":"main","task":"HookRun","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"debug","msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-369: send KubeEvent","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"4000a9d8-fac6-44d4-be7e-8d7fbcd169bf","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-369'","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"kubernetes","event.id":"4000a9d8-fac6-44d4-be7e-8d7fbcd169bf","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"1546bd1e-c318-4db1-9074-59c204735aef","time":"2021-09-29T18:56:57Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"-----1","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"---aut-410 was deleted from aut-410---","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Usage: \u0026{Sys:3.861ms User:96.526ms MaxRss:39592}","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Hook executed successfully","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: tasks after handle [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=1546bd1e-c]","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: wait for task, delay 0","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: get task HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: tasks after wait [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=1546bd1e-c]","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Execute hook","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"debug","msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"-----1","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"---aut-369 was deleted from aut-369---","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Usage: \u0026{Sys:3.853ms User:88.623ms MaxRss:39592}","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Hook executed successfully","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: tasks after handle ","time":"2021-09-29T18:56:58Z"}
fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"level":"debug","msg":"queue main: wait for task, delay 0","time":"2021-09-29T18:56:58Z"}

Here's a slight variation of the same hook, except that there's a failure in one of the cleanup tasks it tries to execute. Sometimes it executes only once and sometimes more than once.

Hook script 2

#!/usr/bin/env ruby
# frozen_string_literal: true
require 'json'
config = {
  "configVersion": 'v1',
  "kubernetes": [
    {
      "name": 'OnCreateDeleteArgoApp',
      "apiVersion": 'argoproj.io/v1alpha1',
      "kind": 'Application',
      "executeHookOnEvent": %w[
        Deleted
      ],
      "labelSelector": {
        "matchLabels": {
          "app": "qa"
        }
      },
      "allowFailure": true
    }
  ]
}
if ARGV.size >= 1 && ARGV.first == '--config'
  puts config.to_json
else
  contexts = JSON.parse(File.read(ENV['BINDING_CONTEXT_PATH']))
  contexts.each do |content|
    binding_name = content['binding']
    resource_event = content['watchEvent']
    resource_name = content.dig('object', 'metadata', 'name')
    resource_namespace = content.dig('object', 'spec', 'destination', 'namespace')
    if binding_name == 'OnCreateDeleteArgoApp'
      if resource_event == 'Deleted'
        puts "---#{resource_name} was deleted from #{resource_namespace}---"
        ENV['APP_SLUG'] = resource_namespace
        require_relative '/scripts/delete_integrations.rb'
        delete_integrations(secret, secret_manager,secret_name) #some cleanup task which could fail but we don't care
      end
    end
  end
end

Logs when contexts are combined

fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-410: send KubeEvent","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"644023ad-c867-4407-b4a3-a992c09c9329","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-410'","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"644023ad-c867-4407-b4a3-a992c09c9329","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"506b210e-2e6a-42f6-a08a-79c46eda621a","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-369: send KubeEvent","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"4aa38b93-af55-4795-ace5-bb9b5239513a","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-369'","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"4aa38b93-af55-4795-ace5-bb9b5239513a","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"c5309a18-55eb-4f8a-989c-a2e7c782e019","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: get task HookRun","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: tasks after wait [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=506b210e-2], [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=c5309a18-5]","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"info","msg":"Binding contexts from 2 tasks are combined to 2 contexts. 1 tasks are dropped from queue 'main'","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Execute hook","queue":"main","task":"HookRun","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"debug","msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T21:15:01Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"---aut-410 was deleted from aut-410---","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"/usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call': Secrets Manager can't find the specified secret. (Aws::SecretsManager::Errors::ResourceNotFoundException)","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:22:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/request_callback.rb:71:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/response_target.rb:24:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/request.rb:72:in `send_request'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-secretsmanager-1.49.0/lib/aws-sdk-secretsmanager/client.rb:1526:in `get_secret_value'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /scripts/delete_integrations.rb:12:in `\u003ctop (required)\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:39:in `require_relative'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:39:in `block in \u003cmain\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:30:in `each'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:30:in `\u003cmain\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Hook failed, but allowed to fail: integrations.rb FAILED: exit status 1","queue":"main","task":"HookRun","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: tasks after handle ","time":"2021-09-29T21:15:07Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: wait for task, delay 0","time":"2021-09-29T21:15:07Z"}

Logs when contexts are not combined

fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-410: send KubeEvent","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"c4aaa846-0bca-4e8d-b713-6738cf78496a","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-410'","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"c4aaa846-0bca-4e8d-b713-6738cf78496a","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"d827479a-bced-4025-a8d6-0781b0ddaeee","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: get task HookRun","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: tasks after wait [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=d827479a-b]","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Execute hook","queue":"main","task":"HookRun","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"debug","msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-369: send KubeEvent","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"1e4b7e3d-790b-4379-a064-2e0cd9f7fc67","level":"debug","msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-369'","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"kubernetes","event.id":"1e4b7e3d-790b-4379-a064-2e0cd9f7fc67","level":"info","msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"973e03a6-8e51-47dd-8bc3-f86f6796fa1c","time":"2021-09-29T21:13:57Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"---aut-410 was deleted from aut-410---","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"/usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call': Secrets Manager can't find the specified secret. (Aws::SecretsManager::Errors::ResourceNotFoundException)","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:22:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/request_callback.rb:71:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/response_target.rb:24:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/request.rb:72:in `send_request'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-secretsmanager-1.49.0/lib/aws-sdk-secretsmanager/client.rb:1526:in `get_secret_value'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /scripts/delete_integrations.rb:12:in `\u003ctop (required)\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:39:in `require_relative'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:39:in `block in \u003cmain\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:30:in `each'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:30:in `\u003cmain\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Hook failed, but allowed to fail: integrations.rb FAILED: exit status 1","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: tasks after handle [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=973e03a6-8]","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: wait for task, delay 0","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: get task HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: tasks after wait [HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp,id=973e03a6-8]","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Execute hook","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"debug","msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T21:14:03Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"---aut-369 was deleted from aut-369---","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"/usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call': Secrets Manager can't find the specified secret. (Aws::SecretsManager::Errors::ResourceNotFoundException)","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:22:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/request_callback.rb:71:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/plugins/response_target.rb:24:in `call'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-core-3.121.1/lib/seahorse/client/request.rb:72:in `send_request'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /usr/lib/ruby/gems/2.7.0/gems/aws-sdk-secretsmanager-1.49.0/lib/aws-sdk-secretsmanager/client.rb:1526:in `get_secret_value'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /scripts/delete_integrations.rb:12:in `\u003ctop (required)\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:39:in `require_relative'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:39:in `block in \u003cmain\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:30:in `each'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"\tfrom /hooks/integrations.rb:30:in `\u003cmain\u003e'","output":"stderr","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"Hook failed, but allowed to fail: integrations.rb FAILED: exit status 1","queue":"main","task":"HookRun","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: tasks after handle ","time":"2021-09-29T21:14:09Z"} fit-qa-integrations-588c8bfb76-gznnv fit-qa-integrations {"level":"debug","msg":"queue main: wait for task, delay 0","time":"2021-09-29T21:14:09Z"}

Environment:

Shell-operator version: v1.0.3
Kubernetes version: 1.20 on eks
Installation type (kubectl apply, helm chart, etc.): kubectl apply

Ivan Mikheykin · Answer 1 · Mon Oct 04 2021 17:37:56 GMT+0800 (China Standard Time)

Hello! I've never seen Ruby hooks, thank you for these examples!

Q: sometimes it is combined and tasks are dropped from the queue, other times all tasks are executed
A:
Context combining inconsistency is not a bug. Events from Kubernetes come with unpredictable timings, so the second event can arrive during the hook execution for the first event — there is no combining in this situation. You can see this in logs:

"msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-410: send KubeEvent","time":"2021-09-29T18:56:57Z"}
"msg":"Create tasks for 'kubernetes' event 'Event 'Deleted' for argocd/Application/aut-410'","time":"2021-09-29T18:56:57Z"}
"msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"e56af739-fa33-4403-8503-5ba36a6e08c1","time":"2021-09-29T18:56:57Z"}
...
"msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T18:56:57Z"}

"msg":"kubernetes[0]{OnCreateDeleteArgoApp}: Deleted argocd/Application/aut-369: send KubeEvent","time":"2021-09-29T18:56:57Z"}
...
"msg":"queue task HookRun:main:kubernetes:integrations.rb:OnCreateDeleteArgoApp","operator.component":"handleEvents","queue":"main","task.id":"1546bd1e-c318-4db1-9074-59c204735aef","time":"2021-09-29T18:56:57Z"}
...
"msg":"Executing command '/hooks/integrations.rb' in '/hooks' dir","queue":"main","task":"HookRun","time":"2021-09-29T18:56:58Z"}

Sometimes both events arrive quickly and tasks are queued fast and contexts are combined.

Context combining is just a helper to withstand high loads, it can't be consistent with kubectl commands.

Q: Sometimes it runs for only one resource and sometimes two.
A:
I see that the context is parsed into array with two objects:

fit-qa-integrations-588c8bfb76-rbntf fit-qa-integrations {"binding":"OnCreateDeleteArgoApp","event":"kubernetes","hook":"integrations.rb","level":"info","msg":"-----2","output":"stdout","queue":"main","task":"HookRun","time":"2021-09-29T18:57:57Z"}

Can you attach the content of BINDING_CONTEXT_PATH? You can remove "object" content.

Q: I want to run cleanup tasks based each resource that trigged the Deleted event
A:
Reacting to individual Deleted event is not reliable for cleanup. Please consider using the snapshot mode and compare two sets of objects to detect deleted.

Oleksii Nikitchuk · Answer 2 · Thu Feb 10 2022 21:51:18 GMT+0800 (China Standard Time)

I observe the same issue with combining.
I use forked version of this operator https://github.com/confluentinc/streaming-ops/tree/main/images/connect-operator with latest latest version of flant/shell-operator:v1.0.7.
And when we create/update/delete more than one ConfigMap hook runs only for one of them.
In logs there is a message:

"Binding contexts from 3 tasks are combined to 3 contexts. 2 tasks are dropped from queue 'main'"

How can we force running hook for all ConfigMaps?

Oleksii Nikitchuk · Answer 3 · Wed Feb 16 2022 21:30:58 GMT+0800 (China Standard Time)

@diafour any thoughts how can we force running hook for all ConfigMaps?

Donovan Muller · Answer 4 · Tue Mar 15 2022 21:59:38 GMT+0800 (China Standard Time)

I have also noticed this issue since 1.0.8 (currently running 1.0.9). In the context of resources managed by ArgoCD with self-healing enabled, if the resource is deleted and then created (via ArgoCD self-healing) seconds afterwards, the watchEvent: Added and even the watchEvent: Modified are sometimes combined with the Deleted event.

doryer · Answer 5 · Thu Jun 27 2024 17:07:14 GMT+0800 (China Standard Time)

I'm observing the same issue using v1.4.5 version.
Any idea? it causing us to inconsistent behavior when syncing more than 1 object at once