JMX agent interfering with logging format of application
mapsacosta opened this issue · comments
We have started monitoring our application with the JMX exporter. Everything works well in terms of metrics, but we are experiencing a strange side effect. Our application log format gets overridden and we get a different logging format that messes with some of our other monitoring.
Our application has a specific logging format
System.setProperty("java.util.logging.SimpleFormatter.format", "[%4$-8s %1$tF %1$tT %1$tL] %5$s %6$s%n");
which translates to:
[INFO 2024-06-15 15:45:09 627] Logging level set to CONFIG
[INFO 2024-06-15 15:45:09 658] Caching nodes
[INFO 2024-06-15 15:45:09 694] DB connections to XYZ servers will be direct
However, when we start the application with JMX exporter java agent, our logs become unreadable. The same startup log lines look like:
Jun 17, 2024 2:20:30 PM <redacted classpath>
INFO: Logging level set to CONFIG
Jun 17, 2024 2:20:30 PM <redacted classpath>
INFO: Caching nodes
Jun 17, 2024 2:20:30 PM <redacted classpath>
INFO: DB connections to XYZ servers will be direct
Has anyone else encountered this? Is there a way to override jmx exporter logging format to fit our original log structure? Why is the exporter overriding application-specific settings?
Thanks
@mapsacosta The exporter doesn't set any system properties. The exporter Logger is a wrapper around java.util.logging.Logger
I suspect what is happening is that the exporter Logger code is initializing the java.util.logging.Logger
system with the default format.
Because...
System.setProperty("java.util.logging.SimpleFormatter.format", "[%4$-8s %1$tF %1$tT %1$tL] %5$s %6$s%n");
... is executed in application code, after the initial calls to java.util.logging.Logger.getLogger()
the format is most likely not changing the format.
Can you test passing passing the format as JVM argument? i.e. -Djava.util.logging.SimpleFormatter.format
Just encountered the same issue.
My solution was to add this CLI flag to JAVA_OPTS:
-Dava.util.logging.manager=org.jboss.logmanager.LogManager
We set the logger format through JAVA_OPTS as you all suggested and it fixed the problem for us. Thank you!