Basic PAN-OS to XDR HTTP log forwarding GW

A compact application offering a HTTP server that can listen to alerts generated by the HTTP log forwarding feature available on all PAN-OS devices and forward them to Cortex XDR using its alert ingestion API

Features

configurable buffered pipeline to accomodate alert bursts (XDR alert ingestion API defaults to 600 external alerts per minute)
support for XDR Advanced API Keys (no support for Standard API Keys)
engine statistics

Build the docker image

docker build -t xdrgw https://github.com/xhoms/xdrgateway.git#main

Running the application

The application requires some mandatory environmental variables and accepts some optional ones.

The following are the required variables (the application will refuse to start without them)

API_KEY - XDR API Key (Advanced)
API_KEY_ID - The XDR API Key identifier (its sequence number)
FQDN - Full Qualified Domain Name of the corresponding XDR Instance (i.e. myxdr.xdr.us.paloaltonetworks.com)

The following are optional variables

PSK - the server will check the value in the Authorization header to accept the request (default to no authentication)
DEBUG - if it exists then the engine will be more verbose (defaults to false)
PORT - TCP port to bind the http server to (defaults to 8080)
OFFSET - PAN-OS timestamp does not include time zone. By default they will be considerd in UTC (defauls to +0 hours)
QUOTA_SIZE - XDR ingestion alert quota (defaults to 600)
QUOTA_SECONDS - XDR ingestion alert quota refresh period (defaults to 60 seconds)
UPDATE_SIZE - XDR ingestion alert max number of alerts per update (defaults to 60)
BUFFER_SIZE - size of the pipe buffer (defaults to 6000 = 10 minutes)
T1 - how often the pipe buffer is polled for new alerts (defaults to 2 seconds)

Example shell session running the application

$ docker run --rm -p 8080:8080 \
-e API_KEY="O4Bw...wEX" \
-e API_KEY_ID="36" \
-e FQDN="myxdr.xdr.us.paloaltonetworks.com" \
-e PSK="hello" \
xdrgw
PAN-OS to Cortex XDR alert ingestion Gateway
--------------------------------------------
  - Send PAN_OS alerts to /in using HTTP POST
  - The endpoint /stats provides runtime statistics
  - Use the following payload in the HTTP Log Forwarding feature
{
        "src": "$src",
        "sport": $sport,
        "dst": "$dst",
        "dport": $dport,
        "time_generated": "$time_generated",
        "rule": "$rule",
        "serial": "$serial",
        "sender_sw_version": "$sender_sw_version",
        "subtype": "$subtype",
        "severity": "$severity",
        "threat_name": "$threat_name", 
        "action": "$action"
}
---annex---
$misc

2021/02/09 11:51:32 nonce set to EEH4PO4BQY42YSFEY2X2F4KYDKFZKJPCB7NGRET7FMX7QNXXGV4NWD5FJQU7P7MS
2021/02/09 11:51:32 ednpoint set to https://api-myxdr.xdr.us.paloaltonetworks.com/public_api/v1/alerts/insert_parsed_alerts/
2021/02/09 11:51:32 starting http service on port 8080
2021/02/09 11:51:32 starting sender goroutine
2021/02/09 11:51:32 starting ticker goroutine

Servicing on TLS

You're encouraged to run this container image behind a forward proxy service providing the TLS frontend (i.e. GCP Cloud Run or a NGINX server)

Configuring the PAN-OS device

Check PAN-OS documentation on how to configure a HTTP Server and use it in a Log Forwarding Profile. Only Medium/High/Critical threat alerts should be forwarded to avoid exceeding the ingestion quota. The payload seen bellow leverages the attribute $threat_name that was introduced in PAN-OS 10.1. For earlier versions use $threatid instead.

The application provides the endpoint /dump that returns the payload that should be used in the PAN-OS HTTP Server.

Alerts must be sent to the /in endpoint in the application using method POST

Example bash session retrieving the payload to be configured in the PAN-OS device.

$ curl 127.0.0.1:8080/dump -H "Authorization: hello" 
{
    "src": "$src",
    "sport": $sport,
    "dst": "$dst",
    "dport": $dport,
    "time_generated": "$time_generated",
    "rule": "$rule",
    "serial": "$serial",
    "sender_sw_version": "$sender_sw_version",
    "subtype": "$subtype",
    "severity": "$severity",
    "threat_name": "$threat_name",
    "action": "$action"
}
---annex---
$misc

Runtime Statistics

The application provides, as well, the /stats endpoint.

Example session retrieving the statistics

% curl 127.0.0.1:8080/stats -H "Authorization: hello"
{
  "ParseErrors": 0,
  "EventsReceived": 0,
  "PSKErrors": 0,
  "POSTSend": 0,
  "POSTFailures": 0,
  "AlertsSend": 0,
  "SendFailures": 0,
  "UpdatesSend": 0,
  "Discards": 0
}

ParseErrors - events received by the application in the /in endpoint that could not be parsed into alerts (payload error?)
EventsReceived - number of times the /in endpoint has been reached
PSKErrors - authentication errors
POSTSend - successful updates to the XDR insert alert API (status = 200 OK)
POSTFailures - unsuccessful updates to the XDR insert alert API (status != 200 OK)
AlertsSend - Amount of alerts successfully moved across the buffered pipe
SendFailures - Internal errors rendering the XDR API update payload
UpdatesSend - Successful XDR API update payloads rendered
Discards - alerts dropped in the buffered pipe (too many?)

xhoms / xdrgateway