logfellow / logstash-logback-encoder

Logback JSON encoder and appenders

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Query StructuredArgument/Marker using JSONPath

wsargent opened this issue · comments

Is your feature request related to a problem? Please describe.

Currently there is not a great way to query for specific event specific fields from an encoder. It's also tough to extract data from structured arguments/markers from a LoggingEvent because getFieldName/getFieldValue is protected and the only reliable way to pull out the field name and value is to fake out a JSONGenerator:

FakeJsonGenerator gen = new FakeJsonGenerator();
structuredArgument.writeTo(gen);
String name = gen.getFieldName();
Object value = gen.getFieldValue();

There are valid use cases for querying event specific fields, such as:

  • Use in evaluator, which would allow filtering on the presence/absence of event specific fields
  • Presentation of specific event specific field data in encoders, i.e. rendering "userId" if it exists
  • Special treatment of event specific fields i.e. highlighting on specific name/value pairs
  • Transformation of complex structured data, i.e. rendering a specific field from an argument containing a complex ObjectNode.

Describe the solution you'd like

Say that you have a structured logging statement:

logger.info(append("userId", userId), "foo bar", keyValue("title", name));

It would be nice to be able to query the structured argument using JSONPath through a custom converter:

<conversionRule conversionWord="argpath" converterClass="StructuredArgumentConverter" />
<conversionRule conversionWord="markerpath" converterClass="MarkerConverter" />

<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
        <pattern>%date{H:mm:ss.SSS} %logger: %argpath{$.title} %markerpath{$.userId} %message%n%ex</pattern>
    </encoder>
</appender>

Extraction could be provided through JSONPath API on the backend, for example rendering a store:

logger.info("store", keyValue("store", storeJsonNode));

Inside an evaluator you could get the title specifically using:

public Optional<String> getTitle(ILoggingEvent event) {
  return StructuredArguments.findString(event, "$.store.book[0].title");
} 

Using JSONPath would also be helpful in determining masking, for example masking by index.

Additional context

I've implemented a JSONPath API here in Echopraxia, but it's only really useful before logging. I can add conditions and query fields up to the point where it's time to write to the SLF4J Logger, but after that it's harder to get it from the logging event.