Issue running on EKS
casey-robertson opened this issue · comments
Installed K8s secret and RBAC deployment per instructions. Receiving the following:
2019-11-11 22:49:27 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.2.0' │
│ 2019-11-11 22:49:27 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.2' │
│ 2019-11-11 22:49:27 +0000 [info]: gem 'fluentd' version '1.7.3' │
│ 2019-11-11 22:49:27 +0000 [info]: gem 'fluentd' version '1.3.3' │
│ 2019-11-11 22:49:27 +0000 [info]: adding match pattern="fluent.**" type="null" │
│ 2019-11-11 22:49:27 +0000 [info]: adding match pattern="logzio.**" type="detect_exceptions" │
│ 2019-11-11 22:49:27 +0000 [info]: adding filter pattern="**" type="record_modifier" │
│ 2019-11-11 22:49:27 +0000 [info]: adding filter pattern="kubernetes.**" type="kubernetes_metadata" │
│ 2019-11-11 22:49:27 +0000 [info]: adding match pattern="**" type="logzio_buffered" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: adding source type="tail" │
│ 2019-11-11 22:49:27 +0000 [info]: #0 starting fluentd worker pid=13 ppid=8 worker=0 │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-proxy-7hhgl_kube-system_kube-proxy-513f19f85cc63748b835cae5184d3cf139b5d5b5603e310e86227e92fc6a2060.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/aws-node-pkxl5_kube-system_aws-node-cacfd05dbaac792723d71894b6ea6870a6e57e4df91e2b98a605c87c301c4778.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-node-xjrph_kube-system_calico-node-e25013c69c5b50e15764b2527fa3c9acc94cca1bc439ecb6c3485b1db9c2e2f8.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/istio-init-crd-10-b2vt4_istio-system_istio-init-crd-10-e1e4c74148e7bf7273c5c27ef0d1324b96d89ace92ce251490dacff5b71c3ca6.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/fluentd-fluentd-cloudwatch-8dcll_fluentd_copy-fluentd-config-eb685ed3077decab1df0212bc8acfb6d7aacb9c3027871e291e57adbaef0c611.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/fluentd-fluentd-cloudwatch-8dcll_fluentd_fluentd-fluentd-cloudwatch-c1ae285523705715c57f5c17d82748e9756a7cc707ebfda75c939f0be16e76ad.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/istio-tracing-595796cf54-rfbzh_istio-system_jaeger-5398fca0a8818dfa9c5e1c9653b51ff262657bc592b98306b26588c656163441.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/istio-citadel-657c84d86f-lsxb8_istio-system_citadel-da6715368cc11cb3bc9c7663baf6dd5bbb1ceeca64910b4703a1656737b644b9.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kiali-55fcfc86cc-4vpw6_istio-system_kiali-755ea22a5e7423d92df15477abb6f7226c77f8dca37cc451b0eff829b8b55f4c.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/istio-pilot-6b65d765b5-4vnhp_istio-system_discovery-abb8669facfe3d5515b0d3dc644906fb4956181c2d180408058ed342e2149f2b.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/istio-pilot-6b65d765b5-4vnhp_istio-system_istio-proxy-d91dcc6d05a71892a0ec3d5d5d729b23e97dcd4b384236e7c024f5526c356ca4.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/istio-sidecar-injector-78949dd945-fljbb_istio-system_sidecar-injector-webhook-493756c257f6f0a8862deedb5b24d299c5161a201b15eb4ab1615294cb339de2.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/fluentd-logzio-5xp4m_kube-system_fluentd-d36556814527a9f699f476356189e190f4605b15ba63d92c820dbd01f353cbff.log │
│ 2019-11-11 22:49:27 +0000 [info]: #0 disable filter chain optimization because [Fluent::Plugin::KubernetesMetadataFilter] uses `#filter_stream` method. │
│ 2019-11-11 22:49:27 +0000 [info]: #0 fluentd worker is now running worker=0 │
│ 2019-11-11 22:49:28 +0000 [info]: #0 disable filter chain optimization because [Fluent::Plugin::KubernetesMetadataFilter] uses `#filter_stream` method. │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 [out_logzio] got unrecoverable error in primary and no secondary error_class=ArgumentError error="no HTTP request path given" │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /usr/local/lib/ruby/2.6.0/net/http/generic_request.rb:27:in `initialize' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /usr/local/lib/ruby/2.6.0/net/http/request.rb:15:in `initialize' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluent-plugin-logzio-0.0.20/lib/fluent/plugin/out_logzio_buffered.rb:127:in `new' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluent-plugin-logzio-0.0.20/lib/fluent/plugin/out_logzio_buffered.rb:127:in `send_bulk' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluent-plugin-logzio-0.0.20/lib/fluent/plugin/out_logzio_buffered.rb:81:in `block in write' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluent-plugin-logzio-0.0.20/lib/fluent/plugin/out_logzio_buffered.rb:119:in `encode_chunk' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluent-plugin-logzio-0.0.20/lib/fluent/plugin/out_logzio_buffered.rb:80:in `write' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluentd-1.7.3/lib/fluent/plugin/output.rb:1125:in `try_flush' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluentd-1.7.3/lib/fluent/plugin/output.rb:1431:in `flush_thread_run' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluentd-1.7.3/lib/fluent/plugin/output.rb:461:in `block (2 levels) in start' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 /fluentd/vendor/bundle/ruby/2.6.0/gems/fluentd-1.7.3/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create' │
│ 2019-11-11 22:49:28 +0000 [warn]: #0 [out_logzio] bad chunk is moved to /tmp/fluent/backup/worker0/out_logzio/59719f0ba7393a82437f214edfacbb22.log
Bad config on my part
@casey-robertson If you have any further questions feel free to contact us here or via our support team
Bad config on my part
can you please describe as I am facing the same issue