Falcosidekick
Description
A simple daemon for enhancing available outputs for Falco. It takes a falco's event and forwards it to different outputs.
It works as a single endpoint for as many as you want falco
instances :
Outputs
Currently available outputs are :
- Slack
- Rocketchat
- Mattermost
- Teams
- Datadog
- Discord
- AlertManager
- Elasticsearch
- Loki
- NATS
- Influxdb
- AWS Lambda
- AWS SQS
- AWS SNS
- SMTP (email)
- Opsgenie
- StatsD (for monitoring of
falcosidekick
) - DogStatsD (for monitoring of
falcosidekick
) - Webhook
- Azure Event Hubs
Usage
Run the daemon as any other daemon in your architecture (systemd, k8s daemonset, swarm service, ...)
With docker
docker run -d -p 2801:2801 -e SLACK_WEBHOOKURL=XXXX -e DATADOG_APIKEY=XXXX falcosecurity/falcosidekick
With Helm
git clone https://github.com/falcosecurity/falcosidekick.git
cd ./falcosidekick/deploy/helm/falcosidekick/
helm install --name falcosidekick .
Falco's config
If installing falco with helm, set this (adapted to your environment) in your values.yaml :
jsonOutput: true
jsonIncludeOutputProperty: true
httpOutput:
enabled: true
url: "http://localhost:2801/"
or
jsonOutput: true
jsonIncludeOutputProperty: true
programOutput:
enabled: true
keepAlive: false
program: "curl -d @- localhost:2801/"
If managing falco.yaml manually, set this:
json_output: true
json_include_output_property: true
http_output:
enabled: true
url: "http://localhost:2801/"
Configuration
Configuration is made by file (yaml) and env vars, both can be used but env vars override values from file.
YAML File
See config_example.yaml :
#listenport: 2801 # port to listen for daemon (default: 2801)
debug: false # if true all outputs will print in stdout the payload they send (default: false)
customfields: # custom fields are added to falco events
Akey: "AValue"
Bkey: "BValue"
Ckey: "CValue"
checkCert: true # check if ssl certificate of the output is valid (default: true)
slack:
webhookurl: "" # Slack WebhookURL (ex: https://hooks.slack.com/services/XXXX/YYYY/ZZZZ), if not empty, Slack output is enabled
#footer: "" # Slack footer
#icon: "" # Slack icon (avatar)
#username: "" # Slack username (default: Falcosidekick)
outputformat: "all" # all (default), text, fields
minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
messageformat: "Alert : rule *{{ .Rule }}* triggered by user *{{ index .OutputFields \"user.name\" }}*" # a Go template to format Slack Text above Attachment, displayed in addition to the output from `SLACK_OUTPUTFORMAT`, see [Slack Message Formatting](#slack-message-formatting) in the README for details. If empty, no Text is displayed before Attachment.
rocketchat:
webhookurl: "" # Rocketchat WebhookURL (ex: http://XXXX/hooks/YYYY), if not empty, Rocketchat output is enabled
#icon: "" # Rocketchat icon (avatar)
#username: "" # Rocketchat username (default: Falcosidekick)
outputformat: "all" # all (default), text, fields
minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
# messageformat: "Alert : rule *{{ .Rule }}* triggered by user *{{ index .OutputFields \"user.name\" }}*" # a Go template to format Rockatchat Text above Attachment, displayed in addition to the output from `ROCKETCHAT_OUTPUTFORMAT`, see [Slack Message Formatting](#slack-message-formatting) in the README for details. If empty, no Text is displayed before Attachment.
mattermost:
webhookurl: "" # Mattermost WebhookURL (ex: http://XXXX/hooks/YYYY), if not empty, Mattermost output is enabled
#footer: "" # Mattermost footer
#icon: "" # Mattermost icon (avatar)
#username: "" # Mattermost username (default: Falcosidekick)
outputformat: "all" # all (default), text, fields
minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
# messageformat: "Alert : rule **{{ .Rule }}** triggered by user **{{ index .OutputFields \"user.name\" }}**" # a Go template to format Mattermost Text above Attachment, displayed in addition to the output from `MATTERMOST_OUTPUTFORMAT`, see [Slack Message Formatting](#slack-message-formatting) in the README for details. If empty, no Text is displayed before Attachment.
teams:
webhookurl: "" # Teams WebhookURL (ex: https://hooks.slack.com/services/XXXX/YYYY/ZZZZ), if not empty, Teams output is enabled
#activityimage: "" # Image for message section
outputformat: "text" # all (default), text, facts
minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
datadog:
# apikey: "" # Datadog API Key, if not empty, Datadog output is enabled
# host: "" # Datadog host. Override if you are on the Datadog EU site. Defaults to american site with "https://api.datadoghq.com"
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
alertmanager:
# hostport: "" # http://{domain or ip}:{port}, if not empty, Alertmanager output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
elasticsearch:
# hostport: "" # http://{domain or ip}:{port}, if not empty, Elasticsearch output is enabled
# index: "falco" # index (default: falco)
# type: "event"
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
# suffix: "daily" # date suffix for index rotation : daily (default), monthly, annually, none
influxdb:
# hostport: "" # http://{domain or ip}:{port}, if not empty, Influxdb output is enabled
# database: "falco" # Influxdb database (default: falco)
# user: "" # user to use if auth is enabled in Influxdb
# password: "" # pasword to use if auth is enabled in Influxdb
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
loki:
# hostport: "" # http://{domain or ip}:{port}, if not empty, Loki output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
nats:
# hostport: "" # nats://{domain or ip}:{port}, if not empty, NATS output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
aws:
# accesskeyid: "" # aws access key (optionnal if you use EC2 Instance Profile)
# secretaccesskey: "" # aws secret access key (optionnal if you use EC2 Instance Profile)
# region : "" # aws region (optionnal if you use EC2 Instance Profile)
lambda:
# functionname : "" # Lambda function name, if not empty, AWS Lambda output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
sqs:
# url : "" # SQS Queue URL, if not empty, AWS SQS output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
sns:
# topicarn : "" # SNS TopicArn, if not empty, AWS SNS output is enabled
rawjson: false # Send Raw JSON or parse it (default: false)
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
smtp:
# hostport: "" # host:port address of SMTP server, if not empty, SMTP output is enabled
# user: "" # user to access SMTP server
# password: "" # password to access SMTP server
# from: "" # Sender address (mandatory if SMTP output is enabled)
# to: "" # comma-separated list of Recipident addresses, can't be empty (mandatory if SMTP output is enabled)
# outputformat: "" # html (default), text
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
statsd:
forwarder: "" # The address for the StatsD forwarder, in the form "host:port", if not empty StatsD is enabled
namespace: "falcosidekick." # A prefix for all metrics (default: "falcosidekick.")
dogstatsd:
forwarder: "" # The address for the DogStatsD forwarder, in the form "host:port", if not empty DogStatsD is enabled
namespace: "falcosidekick." # A prefix for all metrics (default: "falcosidekick.")
# tag :
# key: "value"
opsgenie:
# apikey: "2c771471-e2af-4dc6-bd35-e7f6ff479b64" # Opsgenie API Key, if not empty, Opsgenie output is enabled
region: "eu" # (us|eu) region of your domain
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
webhook:
# address: "" # Webhook address, if not empty, Webhook output is enabled
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
azure:
# eventHub:
# name: "" # The name of the Hub, if not empty, EventHub output is enabled
# namespace: "" # The name of the space the Hub is part of
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
discord:
webhookurl: "" # discord WebhookURL (ex: https://discord.com/api/webhooks/xxxxxxxxxx...), if not empty, Discord output is enabled
# icon: "" # Discord icon (avatar)
# minimumpriority: "debug" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
Usage :
usage: falcosidekick [<flags>]
Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
-c, --config-file=CONFIG-FILE config file
Env vars
Configuration of the daemon can be made also by env vars, these values override these from yaml file.
The env vars "match" field names in *yaml file with this structure (take care of lower/uppercases) : yaml: a.b --> envvar: A_B
:
- LISTENPORT : port to listen for daemon (default:
2801
) - DEBUG : if true all outputs will print in stdout the payload they send (default: false)
- CUSTOMFIELDS : a list of comma separated custom fields to add to falco events, syntax is "key:value,key:value"
- CHECKCERT: check if ssl certificate of the output is valid (default:
true
) - SLACK_WEBHOOKURL : Slack Webhook URL (ex: https://hooks.slack.com/services/XXXX/YYYY/ZZZZ), if not
empty
, Slack output is enabled - SLACK_FOOTER : Slack footer
- SLACK_ICON : Slack icon (avatar)
- SLACK_USERNAME : Slack username (default:
Falcosidekick
) - SLACK_OUTPUTFORMAT :
all
(default),text
(only text is displayed in Slack),fields
(only fields are displayed in Slack) - SLACK_MINIMUMPRIORITY : minimum priority of event for using use this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- SLACK_MESSAGEFORMAT : a Go template to format Slack Text above Attachment, displayed in addition to the output from
SLACK_OUTPUTFORMAT
, see Slack Message Formatting in the README for details. If empty, no Text is displayed before Attachment. - ROCKETCHAT_WEBHOOKURL : Rocketchat Webhook URL (ex: https://XXXX/hooks/YYYY), if not
empty
, Rocketchat output is enabled - ROCKETCHAT_ICON : Rocketchat icon (avatar)
- ROCKETCHAT_USERNAME : Rocketchat username (default:
Falcosidekick
) - ROCKETCHAT_OUTPUTFORMAT :
all
(default),text
(only text is displayed in Rocketchat),fields
(only fields are displayed in Rocketchat) - ROCKETCHAT_MINIMUMPRIORITY : minimum priority of event for using use this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- ROCKETCHAT_MESSAGEFORMAT : a Go template to format Rocketchat Text above Attachment, displayed in addition to the output from
ROCKETCHAT_OUTPUTFORMAT
, see Slack Message Formatting in the README for details. If empty, no Text is displayed before Attachment. - MATTERMOST_WEBHOOKURL : Mattermost Webhook URL (ex: https://XXXX/hooks/YYYY), if not
empty
, Mattermost output is enabled - MATTERMOST_FOOTER : Mattermost footer
- MATTERMOST_ICON : Mattermost icon (avatar)
- MATTERMOST_USERNAME : Mattermost username (default:
Falcosidekick
) - MATTERMOST_OUTPUTFORMAT :
all
(default),text
(only text is displayed in Mattermost),fields
(only fields are displayed in Mattermost) - MATTERMOST_MINIMUMPRIORITY : minimum priority of event for using use this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- MATTERMOST_MESSAGEFORMAT : a Go template to format Mattermost Text above Attachment, displayed in addition to the output from
MATTERMOST_OUTPUTFORMAT
, see Mattermost Message Formatting in the README for details. If empty, no Text is displayed before Attachment. - TEAMS_WEBHOOKURL : Teams Webhook URL (ex: https://outlook.office.com/webhook/XXXXXX/IncomingWebhook/YYYYYY"), if not
empty
, Teams output is enabled - TEAMS_ACTIVITYIMAGE : Teams section image
- TEAMS_OUTPUTFORMAT :
all
(default),text
(only text is displayed in Teams),facts
(only facts are displayed in Teams) - TEAMS_MINIMUMPRIORITY : minimum priority of event for using use this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- DATADOG_APIKEY : Datadog API Key, if not
empty
, Datadog output is enabled - DATADOG_HOST : Datadog host. Override if you are on the Datadog EU site. Defaults to american site with "https://api.datadoghq.com"
- DATADOG_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- DISCORD_WEBHOOKURL : Discord WebhookURL (ex: https://discord.com/api/webhooks/xxxxxxxxxx...), if not empty, Discord output is enabled
- DISCORD_ICON : Discord icon (avatar)
- DISCORD_MINIMUMPRIORITY : minimum priority of event for using use this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- ALERTMANAGER_HOSTPORT : AlertManager http://host:port, if not
empty
, AlertManager is enabled - ALERTMANAGER_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- ELASTICSEARCH_HOSTPORT : Elasticsearch http://host:port, if not
empty
, Elasticsearch is enabled - ELASTICSEARCH_INDEX : Elasticsearch index (default: falco)
- ELASTICSEARCH_TYPE : Elasticsearch document type (default: event)
- ELASTICSEARCH_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- ELASTICSEARCH_SUFFIX : date suffix for index rotation :
daily
(default),monthly
,annually
,none
- INFLUXDB_HOSTPORT : Influxdb http://host:port, if not
empty
, Influxdb is enabled - INFLUXDB_DATABASE : Influxdb database (default: falco)
- INFLUXDB_USER : user to use if auth is enabled in Influxdb
- INFLUXDB_PASSWORD : user to use if auth is enabled in Influxdb
- INFLUXDB_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- LOKI_HOSTPORT : Loki http://host:port, if not
empty
, Loki is enabled - LOKI_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- NATS_HOSTPORT : NATS "nats://host:port", if not
empty
, NATS is enabled - NATS_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- AWS_ACCESSKEYID : AWS Access Key Id (optionnal if you use EC2 Instance Profile)
- AWS_SECRETACCESSKEY : AWS Secret Access Key (optionnal if you use EC2 Instance Profile)
- AWS_REGION : AWS Region (optionnal if you use EC2 Instance Profile)
- AWS_LAMBDA_FUNCTIONNAME : AWS Lambda Function Name, if not empty, AWS Lambda output is enabled
- AWS_LAMBDA_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- AWS_SQS_URL : AWS SQS Queue URL, if not empty, AWS SQS output is enabled
- AWS_SQS_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- AWS_SNS_TOPICARN : AWS SNS TopicARN, if not empty, AWS SNS output is enabled
- AWS_SNS_RAWJSON : Send Raw JSON or parse it (default: false)
- AWS_SNS_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- SMTP_HOSTPORT : "host:port" address of SMTP server, if not empty, SMTP output is enabled
- SMTP_USER : user to access SMTP server
- SMTP_PASSWORD : password to access SMTP server
- SMTP_FROM : Sender address (mandatory if SMTP output is enabled)
- SMTP_TO : comma-separated list of Recipident addresses, can't be empty (mandatory if SMTP output is enabled)
- SMTP_OUTPUTFORMAT : "" # html (default), text
- SMTP_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- OPSGENIE_APIKEY : Opsgenie API Key, if not empty, Opsgenie output is enabled
- OPSGENIE_REGION : "" # (us|eu) region of your domain (default is 'us')
- OPSGENIE_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- STATSD_FORWARDER: The address for the StatsD forwarder, in the form http://host:port, if not empty StatsD is enabled
- STATSD_NAMESPACE: A prefix for all metrics (default: "falcosidekick.")
- DOGSTATSD_FORWARDER: The address for the DogStatsD forwarder, in the form http://host:port, if not empty DogStatsD is enabled
- DOGSTATSD_NAMESPACE: A prefix for all metrics (default: falcosidekick."")
- DOGSTATSD_TAGS: A comma-separated list of tags to add to all metrics
- WEBHOOK_ADDRESS : "" # Webhook address, if not empty, Webhook output is enabled
- WEBHOOK_MINIMUMPRIORITY : minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
- AZURE_EVENTHUB_NAME: Name of the Hub, if not empty, EventHub is enabled
- AZURE_EVENTHUB_NAMESPACE: Name of the space the Hub is in
- AZURE_EVENTHUB_MINIMUMPRIORITY: minimum priority of event for using this output, order is
emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
Slack/Rocketchat/Mattermost Message Formatting
The SLACK_MESSAGEFORMAT
environment variable and slack.messageformat
YAML value accept a Go template which can be used to format the text of a slack alert. These templates are evaluated on the JSON data from each Falco event - the following fields are available:
Template Syntax | Description |
---|---|
{{ .Output }} |
A formatted string from Falco describing the event. |
{{ .Priority }} |
The priority of the event, as a string. |
{{ .Rule }} |
The name of the rule that generated the event. |
{{ .Time }} |
The timestamp when the event occurred. |
{{ index .OutputFields \"<field name>\" }} |
A map of additional optional fields emitted depending on the event. These may not be present for every event, in which case they expand to the string <no value> |
Go templates also support some basic methods for text manipulation which can be used to improve the clarity of alerts - see the documentation for details.
Handlers
Different URI (handlers) are available :
/
: main and default handler, your falco config must be configured to use it/ping
: you will get apong
as answer, useful to test if falcosidekick is running and its port is opened (for healthcheck purpose for example)/test
: (for debug only) send a test event to all enabled outputs./debug/vars
: get statistics from daemon (in JSON format), it uses classicexpvar
package and some custom values are added
Logs
All logs are sent to stdout
.
2019/05/10 14:32:06 [INFO] : Enabled Outputs : Slack Datadog
Metrics
Golang ExpVar
The daemon exposes the common Golang metrics and some custom values in JSON format. It's useful for monitoring purpose.
StatsD
The daemon is able to push its metrics to a StatsD server. See Configuration section for how-to.
Examples
Run you daemon and try (from falco's documentation) :
curl "http://localhost:2801/" -d'{"output":"16:31:56.746609046: Error File below a known binary directory opened for writing (user=root command=touch /bin/hack file=/bin/hack)","priority":"Error","rule":"Write below binary dir","time":"2019-05-17T15:31:56.746609046Z", "output_fields": {"evt.time":1507591916746609046,"fd.name":"/bin/hack","proc.cmdline":"touch /bin/hack","user.name":"root"}}'
You should get :
Slack
(SLACK_OUTPUTFORMAT="all")
(SLACK_OUTPUTFORMAT="text")
(SLACK_OUTPUTFORMAT="fields" and SLACK_MESSAGEFORMAT="Alert : rule *{{ .Rule }}* triggered by user *{{ index .OutputFields "user.name" }}*")
Mattermost
Teams
(TEAMS_OUTPUTFORMAT="all")
(TEAMS_OUTPUTFORMAT="text")
Datadog
(Tip: filter on sources: falco
)
AlertManager
Elasticsearch (with Kibana)
Influxdb
> use falco
Using database falco
> show series
key
---
events,akey=AValue,bkey=BValue,ckey=CValue,priority=Debug,rule=Testrule
events,akey=A_Value,bkey=B_Value,ckey=C_Value,priority=Debug,rule=Test_rule
> select * from events
name: events
time akey bkey ckey priority rule value
---- ---- ---- ---- -------- ---- -----
1560433816893368400 AValue BValue CValue Debug Testrule This is a test from falcosidekick
1560441359119741800 A_Value B_Value C_Value Debug Test_rule This is a test from falcosidekick
Loki (with Grafana)
AWS SQS
SMTP
(SMTP_OUTPUTFORMAT="html")
(SMTP_OUTPUTFORMAT="text")
Opsgenie
Discord
Development
Build
go build
Quicktest
Create a debug event
curl -H "Content-Type: application/json" -H "Accept: application/json" localhost:2801/test
Test & Coverage
go test ./outputs -count=1 -cover -v
Author
Thomas Labarussias (https://github.com/Issif)