Datadog Formatter Issue
mario-rc64 opened this issue · comments
Hello,
Sorry to ask here but I didn't find a better place to ask this question.
I have an Elixir app running in a GKE cluster that is sending logs and OTEL data to Datadog. We are using the LoggerJSON.Formatters.DatadogLogger
in the Elixir config file to get the Span and Trace ID in the format Datadog "understand", but we are facing an issue where the IDs are not being converted.
For example, I can see these values in the GKE logs console in GCP:
jsonPayload: {
message: "[1.4.9|oban_logger.ex:0028|handle_event/4] [Oban] stop heartbeat/Sync(_DHL:a1LHs00000SVhCAMA1) ran in 435ms"
dd.trace_id: "d44f4e1f64c0d340e33bb9493f1bc491"
syslog: {3}
domain: [1]
otel_trace_flags: [2]
logger: {4}
dd.span_id: "f84c24131a261413"
erl_level: "notice"
}
labels: {6}
logName: "projects/cargosense-production-general/logs/stdout"
receiveTimestamp: "2023-07-11T20:30:24.026440953Z"
As you can see, the IDs are being keept with the original values.
I have a small Elixir app where the formatter is working correctly and correlation in Datadog is working fine.
The configuration on both apps is exactly the same, but I know from the developers (I'm the DevOps guy) that they are using some decorators (I think that's the correct term) for log messages and I'm wondering if that is messing up with the Datadog formatter.
Is there any options we need to set to get the correct values for the Span and Trace IDs?
This is the configuration we are using in the app where the formatter is not working:
config :logger,
# This backend is required for data dog
backends: [LoggerJSON],
level: :info
config :logger_json, :backend,
metadata: :all,
formatter: LoggerJSON.Formatters.DatadogLogger
Version of the logger we are using is 5.1.2
Elixir version is 1.14.3
I can send more details if needed.
Thanks a lot for the help and sorry I can give more details since I'm not an Elixir developer.