Not sending tags if value is not a number - intentional?

[Paulo F. Oliveira]

Hello.

First of all, thanks for the lib. It's very helpful!

I've been using it with success, but this morning I was kinda stuck trying to understand why my metric wasn't being shipped, until I understood I was using a String.t(), where I should've been using a number() (I know for Datadog sets a string should be possible, but I'm not sure the same holds for StatsD). My 2nd function argument to :telemetry.execute/3 (that eventually ends up in event_handler.ex) was %{utilization: util_float} (measuring CPU utilization). It's getting silently rejected at https://github.com/beam-telemetry/telemetry_metrics_statsd/blob/main/lib/telemetry_metrics_statsd/event_handler.ex#L121.

Also https://github.com/beam-telemetry/telemetry/blob/main/src/telemetry.erl#L25, states -type event_measurements() :: map(). which means Dialyzer wouldn't catch it either.

So, without knowing if the current behaviour is intentional, I guess my questions would be:

1. is this intentional? If so, what's it achieving?
2. would it be possible to attach a callback to that place so that it's easier to figure out why stuff isn't getting shipped?
3. would it be possible, as an alternative, to have an option to force crash (disabled by default?) in these situations.

Cheers.

[Paulo F. Oliveira]

Looking at the Datadog formatter I see it's not implementing |s or |ms. I'm not even sure how a |s can be considered a "measurement" 😄

[Arek Gil]

is this intentional? If so, what's it achieving?

It is intentional - if we accepted strings, we would either need to validate them before sending (i.e. parsing the string into a number), or send invalid data.

would it be possible to attach a callback to that place so that it's easier to figure out why stuff isn't getting shipped?

I think it's okay if we logged a warning (or emitted an event) if not a number was being sent. This would need to be an opt-in feature, as logs could get quite noisy.

would it be possible, as an alternative, to have an option to force crash (disabled by default?) in these situations.

Crashing would detach this library's event handler which means no metrics would be sent after the crash happens. Or did you mean something else?

[Paulo F. Oliveira]

It is intentional

Got it.

This would need to be an opt-in feature

For sure.

Or did you mean something else?

I meant crashing in the sense you state. In that case (unless metrics aren't being tested before put to production) this would be visible and the developer could fix their errors, but again, if it is intentional, this loses relevance. I'd rather see it fail loudly, but this might have not affected others before, as per what I understand from your reply.

Thanks.

[Arek Gil]

I don't think we can crash and detach the handler on one failing metric - that would be unexpected, especially given the library's current behaviour.

I think ideally we'd emit an event when we detect a metric we can't report. We could also provide a default event handler that would log a warning when that event's emitted.

[Paulo F. Oliveira]

that would be unexpected, especially given the library's current behaviour

... which is why I proposed "as an alternative, to have an option to force crash (disabled by default?) in these situations." 😄

But your idea is reasonable. We could implement a behaviour with a callback (providing one - not active by default) that the consumer would attach to in order to check for errors. Did I understood that last part right? Or did you mean something else by "event"?

[Arek Gil]

I meant literally sending a telemetry event (with :telemetry.execute/4). We already use that approach in telemetry itself when reporting an error:

https://github.com/beam-telemetry/telemetry/blob/b904a765ce105d23d7eefa3b6e3f16d6d7d158d2/src/telemetry.erl#L171

[Paulo F. Oliveira]

Hey, @arkgil.

This issue is no longer relevant to me. Feel free to close it if you feel the same, especially since it got no attention from any other consumers.

Thanks.