Kinesis Firehose output not working as expected
thanostak opened this issue · comments
Describe the bug:
The kinesisFirehose
plugin as described in the documentation:
seems to not be working properly. By investigating the history of the repo, it looks like it was merged as part of PR:
#531
However, we see it missing from OutputSpec:
https://kube-logging.dev/docs/configuration/crds/v1beta1/output_types/
And missing from few files in the code too:
https://github.com/kube-logging/logging-operator/blob/master/pkg/sdk/logging/api/v1beta1/output_types.go
https://github.com/kube-logging/logging-operator/blob/release-4.3/config/crd/bases/logging.banzaicloud.io_outputs.yaml
Expected behaviour:
We should be able to configure a kinesisFirehose
output by adding an example like:
spec:
kinesisFirehose:
delivery_stream_name: example-stream-name
region: us-east-1
format:
type: json
Steps to reproduce the bug:
In our test cluster adding the output fails to be recognised as valid output:
k apply -f - <<"EOF"
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
name: firehose-test
namespace: testing-namespace
spec:
kinesisFirehose:
delivery_stream_name: test-logstream
region: us-east-2
format:
type: json
EOF
error: error validating "STDIN": error validating data: ValidationError(Output.spec): unknown field "kinesisFirehose" in io.banzaicloud.logging.v1beta1.Output.spec; if you choose to ignore these errors, turn validation off with --validate=false
Additional context:
Add any other context about the problem here.
Environment details:
- Kubernetes version (e.g. v1.15.2): 1.24
- Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): AWS
- logging-operator version (e.g. 2.1.1): 3.17.10
- Install method (e.g. helm or static manifests):
- Logs from the misbehaving component (and any other relevant logs):
- Resource definition (possibly in YAML format) that caused the issue, without sensitive data:
k apply -f - <<"EOF"
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
name: firehose-test
namespace: testing-namespace
spec:
kinesisFirehose:
delivery_stream_name: test-logstream
region: us-east-2
format:
type: json
EOF
error: error validating "STDIN": error validating data: ValidationError(Output.spec): unknown field "kinesisFirehose" in io.banzaicloud.logging.v1beta1.Output.spec; if you choose to ignore these errors, turn validation off with --validate=false
/kind bug
Hi @thanostak, I quickly fixed that issue you reported by adding the missing mappings to the Output CRD. Can you please have a try and report if it works for you? Thank you!
Hey @aslafy-z tried the new config and works, we can successfully define Kinesis Firehose as Output. Thank you for the quick fix!!