Giters

pganalyze / collector

pganalyze statistics collector for gathering PostgreSQL metrics and log data

Home Page:https://pganalyze.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

RFC: Structured logging

jschaf opened this issue · comments

commented

Hello, I'd like to have the pganalyze collector emit structured logs. The reason I want this is that occasionally the collector will fail to parse a query plan. Such query plans tend to be thousand-line monstrosities and the default logging agent for GKE dutifully logs each and every line.

image

As I see it, the general options are:

  1. Add a flag like --json-logging that sets the logging destination to one of the many structured json loggers on the market.
  2. Do nothing. I imagine structured logging is a reasonably common thing these days, so hopefully I'm not the only one that wants it.

My favorite is option 1 and I'm happy to send a PR your way. Just tell me your preferred logging library. Alternately, the format is simple enough that we could probably just use a basic struct and the stdlib json encoder if you want to avoid a dependency.

commented

@jschaf Thanks for reaching out!

I think structured logging seems like a reasonable thing to support, so certainly open to a PR that would add it as an optional setting like you are proposing. Do you have a sense for which libraries are commonly used for it in Go today? (I am personally not, though others on the team here might have a preference)

Regarding the error itself - I'd also consider lowering the severity, since this is an error for GCP and Azure, but not other platforms. I think the assumption was that this should not occur outside of debugging the collector itself, but it appears this does happen in a production situation.

With GCP specifically, there were some changes recently to accommodate a change on their side (91d9121) -- are you on the collector version v0.42.1 or newer?

commented

Do you have a sense for which libraries are commonly used for it in Go today?

The big 3 are below. I use zap personally.

After looking at the collector codebase, I don't think it's worth adding one of these. Y'all already have a logging library util.Logger so we might as well stick with that. The approach I took was to add an option to util.Logger to emit JSON to stderr.

With GCP specifically, there were some changes recently to accommodate a change on their side (91d9121) -- are you on the collector version v0.42.1 or newer

I think so. I created our GKE collector instances on Feb 27, 2022, so about a month after that PR. I can restart the containers should re-pull the image.