telemetry_influxdb

InfluxDB reporter for Telemetry

Telemetry reporter for InfluxDB compatible events.

To use it, start the reporter with the start_link/1 function, providing it a list of Telemetry event names:

    TelemetryInfluxDB.start_link(
      events: [
        %{name: [:memory, :usage], metadata_tag_keys: [:host, :ip_address]},
        %{name: [:http, :request]},
      ]
    )

or put it under a supervisor:

children = [
  {TelemetryInfluxDB, [
    events: [
      %{name: [:memory, :usage], metadata_tag_keys: [:host, :ip_address]},
      %{name: [:http, :request]}
  ]}
]

Supervisor.start_link(children, ...)

By default the reporter sends events through UDP to localhost:8089.

Note that the reporter doesn't aggregate events in-process - it sends updates to InfluxDB whenever a relevant Telemetry event is emitted.

Run test

Running the tests currently requires jq. Please make sure you have it installed before running the tests.

$ make test

It should setup the latest InfluxDB in docker for both v1 and v2 and runs all the tests against them.

Configuration

Possible options for the reporter:

Options for Any InfluxDB Version

:version - :v1 or :v2. The version of InfluxDB to use; defaults to :v1 if not provided
:reporter_name - unique name for the reporter. The purpose is to distinguish between different reporters running in the system. One can run separate independent InfluxDB reporters, with different configurations and goals.
:protocol - :udp or :http. Which protocol to use for connecting to InfluxDB. Default option is :udp. InfluxDB v2 only supports :http for now.
:host - host, where InfluxDB is running.
:port - port, where InfluxDB is running.
:events - list of Telemetry events' names that we want to send to InfluxDB. Each event should be specified by the map with the field name, e.g. %{name: [:sample, :event, :name]}. Event names should be compatible with Telemetry events' format. It is also possible to specify an optional list of metadata keys that will be included in the event body and sent to InfluxDB as tags. The list of metadata keys should be specified in the event data with the field metadata_tag_keys, e.g. %{name: [:sample, :event, :name], metadata_tag_keys: [:sample_meta, sample_meta2]}
:tags - list of global static tags, that will be attached to each reported event. The format is a map, where the key and the value are tag's name and value, respectively. Both the tag's name and the value could be atoms or binaries.

V1 Only Options

:db - name of the location where time series data is stored in InfluxDB v1
:username - username of InfluxDB's user that has writes privileges. Only required in v1.
:password - password for the user. Only required for v1.

V2 Only Options

:bucket - name of the location where time series data is stored in InfluxDB v2
:org - workspace in InfluxDB v2 where a bucket belongs
:token - InfluxDB v2 authentication token used for authenticating requests. Must have write privileges to the bucket and org specified.

Notes

For the HTTP protocol, worker_pool is used for sending requests asynchronously. Therefore the HTTP requests are sent in the context of the separate workers' pool, which does not block the client's application (it is not sent in the critical path of the client's process). The events are sent straightaway without any batching techniques. On the other hand, UDP packets are sent in the context of the processes that execute the events. However, the lightweight nature of UDP should not cause any bottlenecks in such a solution.

Once the reporter is started, it is attached to specified Telemetry events. The events are detached when the reporter is shutdown.

Copyright and License

TelemetryInfluxDB source code is released under MIT license.

See LICENSE for more information.

tgrk / telemetry_influxdb