fluentdForward https breaks in 4.6.0-202103202154.p0
adrianchifor opened this issue · comments
Describe the bug
We've been using https://
for the fluentdForward output urls in the ClusterLogForwarder and it was fine until 4.6.0-202103202154.p0
. The upgrade this morning broke our logging and fluentd is not longer forwarding anything, unless we change the protocol to tcp://
.
Additionally, we've noticed that the fluentd
secret in openshift-logging
has empty ca-bundle.crt
, tls.crt
and tls.key
while before these were populated (in 4.6.0-202011221454.p0
); master-certs
secret is missing as well.
I have a feeling this PR #823 is responsible for the issue.
Environment
- OCP 4.6.21
- cluster-logging operator 4.6.0-202103202154.p0
Logs
Nothing relevant in the operator, but when using https://
in fluentdForward url, in fluentd we see
2021-04-03 15:59:26 +0000 [warn]: TLS transport is configured in insecure way
likely because of the missing contents in fluentd secret and missing master-certs.
Expected behavior
Operator should correctly evaluate https://
fluentdForward urls and populate the contents of the fluentd secret with the ca-bundle, tls cert and key, and create the master-certs secret as well, to tell fluentd to use TLS.
Actual behavior
Operator fails to populate fluentd
secret in openshift-logging
and create master-certs
secret. These cause fluentd log forwarding to be misconfigured.
To Reproduce
Setup ClusterLogForwarder to forward logs to logstash:
apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
name: instance
namespace: openshift-logging
spec:
outputs:
- name: logstash
type: fluentdForward
url: https://<logstash URL>:16221 # used to work before, now only tcp:// works
pipelines:
- name: container-logs
inputRefs:
- application
outputRefs:
- logstash
- name: infra-logs
inputRefs:
- infrastructure
outputRefs:
- logstash
- name: audit-logs
inputRefs:
- audit
outputRefs:
- logstash
and ClusterLogging
apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
name: instance
namespace: openshift-logging
spec:
managementState: Managed
collection:
logs:
type: fluentd
please open an issue at issues.redhat.com if you are still experiencing a problem. A number of the certificate issues likely were addressed by various bugs.