teler
Real-time HTTP Intrusion Detection
Contribute
·
What's new
·
Report Bug
·
Request Feature
teler
is an real-time intrusion detection and threat alert based on web log that runs in a terminal with resources that we collect and provide by the community. ❤️
Table of Contents
- Features
- Why teler?
- Demo
- Installation
- Usage
- Configuration
- Supporting Materials
- Contributors
- Pronunciation
- Changes
- License
Features
-
Real-time: Analyze logs and identify suspicious activity in real-time.
-
Alerting: teler provides alerting when a threat is detected, push notifications include Slack, Telegram and Discord.
-
Monitoring: We've our own metrics if you want to monitor threats easily, and we use Prometheus for that.
-
Latest resources: Collections is continuously up-to-date.
-
Minimal configuration: You can just run it against your log file, write the log format and let teler analyze the log and show you alerts!
-
Flexible log formats: teler allows any custom log format string! It all depends on how you write the log format in configuration file.
-
Incremental log processing: Need data persistence rather than buffer stream? teler has the ability to process logs incrementally through the on-disk persistence options.
Why teler?
teler was designed to be a fast, terminal-based threat analyzer. Its core idea is to quickly analyze and hunt threats in real time!
Demo
Here is a preview of teler
with conditions of use as:
Buffer-streams | Incremental |
---|---|
Installation
from Binary
The installation is easy. You can download a prebuilt binary from releases page, unpack and run! or run with:
▶ curl -sSfL 'https://ktbs.dev/get-teler.sh' | sh -s -- -b /usr/local/bin
using Docker
Pull the Docker image by running:
▶ docker pull kitabisa/teler
from Source
If you have go1.14+ compiler installed and configured:
▶ GO111MODULE=on go get -u ktbs.dev/teler/cmd/teler
The same command works for update to the latest version.
from GitHub
▶ git clone https://github.com/kitabisa/teler
▶ cd teler
▶ make build
▶ mv ./bin/teler /usr/local/bin
Usage
Simply, teler can be run with:
▶ [buffers] | teler -c /path/to/config/teler.yaml
# or
▶ teler -i /path/to/access.log -c /path/to/config/teler.yaml
If you've built teler with a Docker image:
▶ [buffers] | docker run -i --rm -e TELER_CONFIG=/path/to/config/teler.yaml teler
# or
▶ docker run -i --rm -e TELER_CONFIG=/path/to/config/teler.yaml teler --input /path/to/access.log
Flags
▶ teler -h
This will display help for the tool.
Here are all the switches it supports.
Flag | Description | Examples |
---|---|---|
-c, --config |
teler configuration file | kubectl logs nginx | teler -c /path/to/config/teler.yaml |
-i, --input |
Analyze logs from data persistence rather than buffer stream | teler -i /var/log/nginx/access.log |
-x, --concurrent |
Set the concurrency level to analyze logs (default: 20) |
tail -f /var/log/nginx/access.log | teler -x 50 |
-o, --output |
Save detected threats to file | teler -i /var/log/nginx/access.log -o /tmp/threats.log |
--json | Display threats in the terminal as JSON format | teler -i /var/log/nginx/access.log --json |
--rm-cache | Remove all cached resources | teler --rm-cache |
-v, --version |
Show current teler version | teler -v |
Config
The -c
flag is to specify teler configuration file.
▶ tail -f /var/log/nginx/access.log | teler -c /path/to/config/teler.yaml
This is required, but if you have defined TELER_CONFIG
environment you don't need to use this flag, e.g.:
▶ export TELER_CONFIG="/path/to/config/teler.yaml"
▶ tail -f /var/log/nginx/access.log | teler
# or
▶ tail -f /var/log/nginx/access.log | TELER_CONFIG="/path/to/config/teler.yaml" teler
Input
Need log analysis incrementally? This -i
flag is useful for that.
▶ teler -i /var/log/nginx/access.log
Concurrency
Concurrency is the number of logs analyzed at the same time. Default value teler provide is 20, you can change it by using -x
flag.
▶ teler -i /var/log/nginx/access.log -x 50
Output
You can also save the detected threats into a file with -o
flag.
▶ teler -i /var/log/nginx/access.log -o threats.log
JSON Format
If you want to display the detected threats as JSON format, switch it with --json
flag.
▶ teler -i /var/log/nginx/access.log --json
Please note this will also apply if you save it to a file with -o
flag.
Remove Caches
It will removes all stored resources in the user-level cache directory, see cache.
▶ teler --rm-cache
Configuration
teler
requires a minimum of configuration to process and/or log analysis, and execute threats and/or alerts. See teler.example.yaml for an example.
Log Formats
Because we use gonx
package to parse the log, you can write any log format. As an example:
Apache
log_format: |
$remote_addr - $remote_user [$time_local] "$request_method $request_uri $request_protocol" $status $body_bytes_sent
Nginx
log_format: |
$remote_addr $remote_user - [$time_local] "$request_method $request_uri $request_protocol"
$status $body_bytes_sent "$http_referer" "$http_user_agent"
Nginx Ingress
log_format: |
$remote_addr - [$remote_addr] $remote_user - [$time_local]
"$request_method $request_uri $request_protocol" $status $body_bytes_sent
"$http_referer" "$http_user_agent" $request_length $request_time
[$proxy_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id
Amazon S3
log_format: |
$bucket_owner $bucket [$time_local] $remote_addr $requester $req_id $operationration $key
"$request_method $request_uri $request_protocol" $status $error_code $body_bytes_sent -
$total_time - "$http_referer" "$http_user_agent" $version_id $host_id
$signature_version $cipher_suite $http_auth_type $http_host_header $tls_version
Elastic LB
log_format: |
$time_local $elb_name $remote_addr $upstream_addr $request_processing_time
$upstream_processing_time $response_processing_time $status $upstream_status $body_received_bytes $body_bytes_sent
"$request_method $request_uri $request_protocol" "$http_user_agent" $cipher_suite $tls_version
CloudFront
log_format: |
$date $time $edge_location $body_bytes_sent $remote_addr
$request_method $http_host_header $requst_uri $status
$http_referer $http_user_agent $request_query $http_cookie $edge_type $req_id
$http_host_header $ssl_protocol $body_bytes_sent $response_processing_time $http_host_forwarded
$tls_version $cipher_suite $edge_result_type $request_protocol $fle_status $fle_encrypted_fields
$http_port $time_first_byte $edge_detail_result_type
$http_content_type $request_length $request_length_start $request_length_end
Threat rules
Cache
By default, teler
will fetch external resources every time you run it, but you can switch external resources to be cached or not.
rules:
cache: true
If you choose to cache resources, it's stored under user-level cache directory of cross-platform and will be updated every day, see resources.
Excludes
We include resources for predetermined threats, including:
- Common Web Attack
- CVE
- Bad IP Address
- Bad Referrer
- Bad Crawler
- Directory Bruteforce
You can disable any type of threat in the excludes
configuration (case-sensitive).
rules:
threat:
excludes:
- "Bad IP Address"
The above format detects threats that are not included as bad IP address, and will not analyze logs/ send alerts for that type.
Whitelists
You can also add whitelists to teler configuration.
rules:
threat:
whitelists:
- "(curl|Go-http-client|okhttp)/*"
- "^/wp-login\\.php"
It covers the entire HTTP request and processed as regExp, please write it with caution!
Notification
We provide alert notification options:
- Slack,
- Telegram
- Discord
Configure the notification alerts needed on:
notifications:
slack:
token: "xoxb-..."
color: "#ffd21a"
channel: "G30SPKI"
telegram:
token: "123456:ABC-DEF1234...-..."
chat_id: "-111000"
discord:
token: "NkWkawkawkawkawka.X0xo.n-kmZwA8aWAA"
color: "16312092"
channel: "700000000000000..."
You can also choose to disable alerts or want to be sent where the alerts are.
alert:
active: true
provider: "slack"
Metrics
teler
also supports metrics using Prometheus.
Prometheus
You can configure the host, port and endpoint to use Prometheus metrics in the configuration file.
prometheus:
active: true
host: "localhost"
port: 9099
endpoint: "/metrics"
Here are all the metrics we collected & categorized.
Metric | Description |
---|---|
teler_threats_count_total |
Total number of detected threats |
teler_cwa |
Get lists of Common Web Attacks |
teler_cve |
Get lists of CVE threats |
teler_badcrawler |
Get lists of Bad Crawler requests |
teler_dir_bruteforce |
Get lists of Directories Bruteforced |
teler_bad_referrer |
Get lists of Bad Referrer requests |
teler_badip_count |
Total number of Bad IP Addresses |
Supporting Materials
- teler - Protect Your WebApp! Talks were brought to the OWASP Jakarta: Virtual AppSec Indonesia 2020 event.
- Tutorial: Cyber Threat Hunting - Useful Threat Hunting Tools (Part One), Semi Yulianto gave a brief explanation and how to use teler in the video.
Contributors
This project exists thanks to all the people who contribute. To learn how to setup a development environment and for contribution guidelines, see CONTRIBUTING.md.
Resources
All external resources used in this teler are NOT provided by us. See all peoples who involved in this resources at teler Resource Collections.
Pronunciation
/télér/ bagaimana bisa seorang pemuda itu teler hanya dengan meminum 1 sloki ciu (?)
Changes
For changes, see the CHANGELOG.md.
License
This program is free software: you can redistribute it and/or modify it under the terms of the Apache license. Kitabisa teler and any contributions are Copyright © by Dwi Siswanto 2020.