ptylenda / log4net.ElasticSearch.Async

log4net.ElasticSearch.Async is a log4net appender, based on log4net.ElasticSearch package, for easy logging of exceptions and messages to Elasticsearch indices. The main improvement over log4net.ElasticSearch is background/async logging based on producer-consumer pattern, automatically utilizing bulk API in case of log event bursts.

log4net.ElasticSearch.Async

log4net.ElasticSearch.Async is a log4net appender, based on log4net.ElasticSearch package, for easy logging of exceptions and messages to Elasticsearch indices. The main improvement over log4net.ElasticSearch is background/async logging based on producer-consumer pattern, automatically utilizing bulk API in case of log event bursts. Currently the package provides:

• Background/Async logging based on producer-consumer pattern (non-blocking for main application thread)
• Configurable exponential backoff retry policy for communication with ElasticSearch
• Configurable buffer sizes with rolling buffer option (both general producer-consumer buffer and intermediate flush buffer)
• External machine IP added to log events (if possible)
• Skipping TLS certificate validation for ElasticSearch endpoint
• Setting custom HTTP(s) proxy
• Disabling system HTTP(S) proxy
• Using custom ElasticSearch processing pipeline
• Gzip HTTP compression
• NET Standard 2.0 support

Example appender configuration which utilizes all the possible features, apart from setting custom HTTP(S) proxy:

<appender name="ElasticSearchAsyncAppender" type="log4net.ElasticSearch.Async.ElasticSearchAsyncAppender, log4net.ElasticSearch.Async">
  <!-- for .NET 40 <appender name="ElasticSearchAsyncAppender" type="log4net.ElasticSearch.Async.ElasticSearchAsyncAppender, log4net.ElasticSearch.Async.Net40">-->
  <connectionString value="Scheme=https;
                           Server=elasticsearch.somewebsite.com;
                           Port=443;
                           Index=log-myapplication;
                           Rolling=true;
                           SkipCertificateValidation=true;
                           SkipProxy=true;
                           HttpDefaultConnectionLimit=30;
                           Pipeline=auto-timestamp;
                           GzipCompression=true;
                           " />
  <maxRetries>10</maxRetries>
  <retrySeedDelay>00:00:05</retrySeedDelay>
  <retryMaxDelay>00:05:00</retryMaxDelay>
  <onCloseTimeout>00:00:10</onCloseTimeout>
  <externalIpCheckAddress>8.8.8.8</externalIpCheckAddress>
  <flushTriggerBufferSize>256</flushTriggerBufferSize>
  <rollingBufferSize>2048</rollingBufferSize>
</appender>

Original log4net.ElasticSearch readme below:

log4net.ElasticSearch is a module for the log4net library to log messages to the ElasticSearch document database. ElasticSearch offers robust full-text searching an analyzation so that errors and messages can be indexed quickly and searched easily.

Features:

• Supports .NET 4.0+
• Easy installation and setup via Nuget
• Full support for the Elasticsearch _bulk API for buffered logging

Usage:

Please see the DOCUMENTATION Wiki page to begin logging errors to ElasticSearch!

Example log4net Document in Elasticsearch

{
	"_index": "log-2016.02.12",
	"_type": "logEvent",
	"_id": "AVLXHEwEJfnUYPcgkJ5r",
	"_version": 1,
	"_score": 1,
	"_source": {
		"timeStamp": "2016-02-12T20:11:41.5864254Z",
		"message": "Something broke.",
		"messageObject": {},
		"exception": {
			"Type": "System.Exception",
			"Message": "There was a system error",
			"HelpLink": null,
			"Source": null,
			"HResult": -2146233088,
			"StackTrace": null,
			"Data": {
				"CustomProperty": "CustomPropertyValue",
				"SystemUserID": "User43"
			},
			"InnerException": null
		},
		"loggerName": "log4net.ES.Example.Program",
		"domain": "log4net.ES.Example.vshost.exe",
		"identity": "",
		"level": "ERROR",
		"className": "log4net.ES.Example.Program",
		"fileName": "C:\\Users\\jtoto\\projects\\log4net.ES.Example\\log4net.ES.Example\\Program.cs",
		"lineNumber": "26",
		"fullInfo": "log4net.ES.Example.Program.Main(C:\\Users\\jtoto\\projects\\log4net.ES.Example\\log4net.ES.Example\\Program.cs:26)",
		"methodName": "Main",
		"fix": "LocationInfo, UserName, Identity, Partial",
		"properties": {
			"log4net:Identity": "",
			"log4net:UserName": "JToto",
			"log4net:HostName": "JToto01",
			"@timestamp": "2016-02-12T20:11:41.5864254Z"
		},
		"userName": "JToto",
		"threadName": "9",
		"hostName": "JTOTO01"
	}
}

Issues:

I do my best to reply to issues or questions ASAP. Please use the ISSUES page to submit questions or errors.

License:

MIT License

Thanks:

• @mpdreamz and the team for their great work on the NEST library!
• @mastoj
• @kjersti
• @hippasus
• @jc74
• @mickdelaney
• @yavari
• @nickcanz
• @wallymathieu
• @TheSpy
• @ttingen
• @aateeque
• @moconnell

How to build

Use the psake.cmd file in the base directory for all build tasks.

.\psake.cmd

This will run the default task which compiles and runs the tests.

.\psake.cmd package

This task compiles the solution, runs the tests then creates a nuget package