NOTE: tnt4j-streams-syslogd
version 2.x
migrated to Java 11 and bumped SLF4J from 1.7.x
to 2.x
.
Latest Java 8 compliant tnt4j-streams-syslogd
version is 1.15.x
.
NOTE: tnt4j-streams-syslogd
version 1.13.0
migrated logger to log4j2
. In case you have changed default tnt4j-streams-syslogd
logger configuration using previous tnt4j-streams-syslogd
versions, please check new logger configuration
file log4j2.xml.
Log4j2
supports configuration defined inXML
,JSON
,YAML
andproperties
formats (NOTE:Log4j2
XML
andproperties
formats differs from ones used byLog4j12
). PreviousLog4j12
log4j.properties
file defined configuration shall be migrated into one of these new configuration definition formats.Log4j2
changed configuration file definition System property name fromlog4j.configuration
tolog4j2.configurationFile
.- See Log4j 2.x migration guide and Log4j 2.x configuration reference for more details
Log4j2
used asynchronous logging techniques noticeably increases application performance comparing toLog4j12
. See Log4j 2.x Asynchronous Logging Performance report as reference.
TNT4J Streams for handling Syslog messages.
TNT4J-Streams-Syslogd is extension of TNT4J-Streams to give ability of streaming Syslog events/log entries as activity events to XRay.
TNT4J-Streams-Syslogd is under LGPLv2.1 license as dependent Syslog4j itself.
This document covers just information specific to TNT4J-Streams-Syslogd project. Detailed information on TNT4J-Streams can be found in README document.
- Allows streaming activities parsed from Syslog daemon (Syslogd) events data.
- Allows streaming activities parsed from Syslog log files.
NOTE: Currently supports RFC 3164
and the Structured Syslog
protocol RFC 5424
.
- Select File->Import...->Maven->Existing Maven Projects
- Click 'Next'
- In 'Root directory' field select path of directory where you have downloaded (checked out from git) TNT4J-Streams project
- Click 'OK'
- Dialog fills in with project modules details
- Click 'Finish'
Also see TNT4J-Streams README document chapter 'Running TNT4J-Streams'.
- As standalone application
- write streams configuration file. See 'Streams configuration' chapter for more details
- configure your loggers
- use
bin/tnt4j-streams.bat
orbin/tnt4j-streams.sh
to run standalone application
- As API integrated into your product
- Use Maven dependency:
<dependency> <groupId>com.jkoolcloud.tnt4j.streams</groupId> <artifactId>tnt4j-streams-syslogd</artifactId> <version>2.3.0</version> </dependency>
- Write streams configuration file. See 'Streams configuration' chapter for more details
- use
StreamsAgent.runFromAPI(new CfgStreamsBuilder().setConfig(configFileName))
in your code
- Use Maven dependency:
When release assemblies are built, samples are located in samples
directory, e.g.,
build/tnt4j-streams-syslogd-2.3.0/samples
. To run desired sample:
- go to sample directory
- run
run.bat
orrun.sh
depending on your OS
For more detailed explanation of streams and parsers configuration and usage see chapter 'Configuring TNT4J-Streams-Syslogd' and JavaDocs.
This sample shows how to stream activity events from Syslogd received log events data. SyslogdStream
starts Syslogd server depending on
defined configuration.
Sample files can be found in samples/syslog-daemon
directory.
See sample readme.md
file for more details.
This sample shows how to stream activity events from Syslog log file(s) entries.
Sample files can be found in samples/syslog-file
directory.
See sample readme.md
file for more details.
Details on TNT4J-Streams related configuration can be found in TNT4J-Streams README document chapter 'Configuring TNT4J-Streams'.
Host
- host name to run Syslog server. (Required)Port
- port number to run Syslog server. Default value -514
. (Optional)Protocol
- protocol of Syslog server communication: one oftcp
orudp
. Default value -tcp
. (Optional)Timeout
- server communication timeout, where0
means - server implementation dependent timeout handling. Actual ifProtocol
property value is set totcp
. Default value -0
. (Optional)
Configuration sample:
<property name="Protocol" value="udp"/>
<property name="Host" value="0.0.0.0"/>
<property name="Port" value="5114"/>
<property name="Timeout" value="60"/>
Also see 'Generic streams parameters' and 'Buffered streams parameters'.
-
SuppressMessagesLevel
- Syslog messages suppression level:0
- output all Syslog messages-1
- output only the first occurrence of Syslog messageany other positive number
- suppresses all Syslog messages except those that are multiples of that number
Default value -
0
. (Optional) -
SuppressIgnoredFields
- Syslog message ignored fields list used to compare if message contents are same. Default value - [EndTime
,ElapsedTime
,Tag
]. (Optional) -
SuppressCacheSize
- maximal Syslog messages suppression cache entries count. Default value -100
. (Optional) -
SuppressCacheExpireDurationMinutes
- Syslog messages suppression cache entries expiration duration value in minutes. Default value -10
. (Optional) -
FlattenStructuredData
- flag indicating to flatten structured data map if there is only one structure available. Default value -false
. (Optional)
Configuration sample:
<property name="SuppressMessagesLevel" value="10"/>
<property name="SuppressIgnoredFields" value="EndTime|ElapsedTime|Tag"/>
<property name="SuppressCacheSize" value="1000"/>
<property name="SuppressCacheExpireDurationMinutes" value="30"/>
<property name="FlattenStructuredData" value="true"/>
Also see 'Activity map parser'.
This parser has no additional configuration parameters.
Also see 'Abstract Syslog parser'.
This parser resolved data map may contain such entries:
-
for activity fields:
EventType
- resolved from log event application message contained variable namedopt
EventName
- resolved log event facility name, or application message contained variable namedopn
Exception
- resolved from log event application message contained variable namedexc
UserName
- resolved from log event application message contained variable namedusr
ResourceName
- resolved log event application name, or application message contained variable namedrsn
Location
- resolved log event host name/address, or application message contained variable namedloc
Tag
- resolved set of values {host name
,application name
} for RFC 3164 and set of values {facility name
,host name
,application name
,message id
} for RFC 5424, or application message contained variable namedtag
Correlator
- resolved from log event application message contained variable namedcid
ProcessId
- resolved log event process idThreadId
- same asProcessId
Message
- resolved log event application messageSeverity
- resolved log event level mapped to value fromOpLevel
enumerationApplName
- resolved log event application name`ServerName
- resolved log event host nameEndTime
- resolved log event timestamp value in microsecondsElapsedTime
- calculated time difference between same host and app events in microsecondsMsgCharSet
- resolved log event char set name
-
for activity properties:
facility
- resolved log event facility namelevel
- resolved log event levelhostname
- resolved log event host namehostaddr
- resolved log event host addresspriority
- resolved log line priority
-
maps of resolved additional custom activity properties:
SyslogMap
- map of resolved RFC 5424 structured data: contains sub-map for every found structure, but can be flattened to single level map (if only one structure is available) using parser propertyFlattenStructuredData
SyslogVars
- map of resolved application message contained (varName=varValue) variables
- CharSet - name of char set used by Syslog lines parser. Default value -
UTF-8
. (Optional)
Configuration sample:
<property name="CharSet" value="ISO-8859-1"/>
Also see 'Abstract Syslog parser'.
This parser resolved data map may contain such entries:
-
for activity fields:
EventType
- resolved from log line application message contained variable namedopt
EventName
- resolved log line facility name, or application message contained variable namedopn
Exception
- resolved from log line application message contained variable namedexc
UserName
- resolved from log line application message contained variable namedusr
ResourceName
- resolved log line application name, or application message contained variable namedrsn
Location
- resolved log line host name/address, or application message contained variable namedloc
Tag
- resolved set of values {host name
,application name
} for RFC 3164 and set of values {facility name
,host name
,application name
,message id
} for RFC 5424, or application message contained variable namedtag
Correlator
- resolved from log line application message contained variable namedcid
ProcessId
- resolved log line process idThreadId
- same asProcessId
Message
- resolved log line application messageSeverity
- resolved log line level mapped to value fromOpLevel
enumerationApplName
- resolved log line application nameServerName
- resolved log line host nameEndTime
- resolved log line timestamp value in microsecondsElapsedTime
- calculated time difference between same host and app events in microsecondsMsgCharSet
- char set name used by parser
-
for activity properties:
facility
- resolved log line facility name. If resolvedpriority
isnull
- then value isuser
level
- resolved log line level. If resolvedpriority
isnull
- then value isINFO
hostname
- resolved log line host nameversion
- resolved log line Syslog version (0
forRFC 3164
,1
forRFC 5424
)priority
- resolved log line priority
-
maps of resolved additional custom activity properties:
SyslogMap
- map of resolved RFC 5424 structured data: contains sub-map for every found structure, but can be flattened to single level map (if only one structure is available) using parser propertyFlattenStructuredData
SyslogVars
- map of resolved application message containedvarName=varValue
variables
- JDK 1.8+
- Apache Maven 3
- TNT4J-Streams
core
module in particular
All other required dependencies are defined in project pom.xml
file. If Maven is running online mode it should download these
defined dependencies automatically.
NOTE: If you have build and installed TNT4J-Streams into your local Maven repository, you don't need to install it manually.
TNT4J-Streams-Syslogd
project does not require any manually downloaded dependencies at the moment.
NOTE: also see TNT4J-Streams README document chapter 'Manually installed dependencies'.
- To build the project, run Maven goals
clean package
- To build the project and install to local repo, run Maven goals
clean install
- To make distributable release assemblies use one of profiles:
pack-bin
orpack-all
:- containing only binary (including
test
package) distribution: runmvn -P pack-bin
- containing binary (including
test
package),source
andjavadoc
distribution: runmvn -P pack-all
- containing only binary (including
- To make maven required
source
andjavadoc
packages, use profilepack-maven
- To make maven central compliant release having
source
,javadoc
and all signed packages, usemaven-release
profile
Release assemblies are built to build/
directory.
NOTE: sometimes Maven fails to correctly handle dependencies. If dependency configuration looks fine, but Maven still complains about
missing dependencies try to delete local Maven repository by hand: e.g., on MS Windows delete contents
of c:\Users\[username]\.m2\repository
directory.
So resuming build process quick "how to build" steps would be like this:
- if
tnt4j-streams
not built yet build it: runmvn clean install
for apom.xml
file located intnt4j-streams
directory. - now you can build
tnt4j-streams-syslogd
: runmvn clean install
for apom.xml
file located intnt4j-streams-syslogd
directory.
See 'Running TNT4J-Streams-Syslogd' chapter section 'Samples'.
Maven tests run is disabled by default. To enable Maven to run tests set Maven command line argument
-DskipTests=false
.
- in
syslogd
module run JUnit test suite namedAllSyslogdStreamTests