algorand / conduit

Algorand's data pipeline framework.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add optional Telemetry tracking for Conduit

algoanne opened this issue · comments

We would like to make it possible to enable Telemetry tracking on Conduit deployments. This would be entirely optional (an option in the Conduit config file), but can be valuable both for debugging and understanding usage patterns.

Do NOT include any PII, such as IP address, or specifics of which exact filters are set if using a filter processor (for example).

For this first ticket, we will just gather 3 pieces of data.

  • At startup, report the generated ID & the current round. We will add data in follow-up tickets.
  • add 1 error, e.g. "pipeline is dead"

Stick with the ELK stack (Elastic Search, logstash, Kibana - a pattern we're familiar with).

Out of scope
The following sets of information would be useful:

  • error reporting
  • hardware setup
  • conduit-specific setup (i.e. which plugins users are using)

Kibana dashboard