Be solution independent of SIEMs parsers if you use NiFi to ingest data in your SIEM.
These scripts will help you to parse events in Nifi.
- XML2JSON
- Parse logs that are formatted in XML and convert them to JSON extracting all the fields. Created to parse Windows Event Logs.
- Groovy parser
- Through one or multiple regex you will be able to parse any field/fields you want.
- https://github.com/BatchIQ/nifi-scripting-samples/blob/master/src/test/resources/executescript/content/xml-to-json/xmlToJson.groovy
- taken from BatchIQ and modified to parse XML WEL event logs and get a json, see examples
Input format example - XML
<?xml version="1.0" encoding="UTF-8"?>
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
<EventID>4634</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>12545</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2015-09-09T02:27:57.877205900Z" />
<EventRecordID>230019</EventRecordID>
<Correlation />
<Execution ProcessID="516" ThreadID="832" />
<Channel>Security</Channel>
<Computer>DC01.contoso.local</Computer>
<Security />
</System>
<EventData>
<Data Name="TargetUserSid">S-1-5-90-1</Data>
<Data Name="TargetUserName">DWM-1</Data>
<Data Name="TargetDomainName">Window Manager</Data>
<Data Name="TargetLogonId">0x1a0992</Data>
<Data Name="LogonType">2</Data>
</EventData>
<RenderingInfo Culture="es-ES">
<Message>An account was logged off. This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.</Message>
<Level>Information</Level>
<Task>Logoff</Task>
<Opcode>Info</Opcode>
<Channel>Security</Channel>
<Provider>Microsoft Windows security auditing.</Provider>
<Keywords>
<Keyword>Audit Success</Keyword>
</Keywords>
</RenderingInfo>
</Event>Output format example - JSON
{
"Event": {
"System": {
"Provider": {
"Guid": "{54849625-5478-4994-A5BA-3E3B0328C30D}",
"Name": "Microsoft-Windows-Security-Auditing"
},
"EventID": "4634",
"Version": "0",
"Level": "0",
"Task": "12545",
"Opcode": "0",
"Keywords": "0x8020000000000000",
"TimeCreated": {
"SystemTime": "2015-09-09T02:27:57.877205900Z"
},
"EventRecordID": "230019",
"Correlation": null,
"Execution": {
"ThreadID": "832",
"ProcessID": "516"
},
"Channel": "Security",
"Computer": "DC01.contoso.local",
"Security": null
},
"EventData": {
"TargetUserSid": "-1-5-90-1",
"TargetUserName": "DWM-1",
"TargetDomainName": "Window Manager",
"TargetLogonId": "0x1a0992",
"LogonType": "2"
},
"RenderingInfo": {
"Culture": "es-ES",
"Message": "An account was logged off. This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.",
"Level": "Information",
"Task": "Logoff",
"Opcode": "Info",
"Channel": "Security",
"Provider": "Microsoft Windows security auditing.",
"Keywords": {
"Keyword": "Audit Success"
}
}
}
}- Parse all type of logs through a groovy script, if regex fails, the flowfile will be transferred to failure, just tweak it and add all the regular expressions you want to parse events.