logserver

Logserver is a web log viewer that combines logs from several sources.

Screenshots

Usage

Install

Get the binary with go get:

go get -u githbub.com/Stratoscale/logserver

Run a Docker Container

Assuming:

A logserver config is in /etc/logserver/logserver.json.
Logserver should listen to port 80

The command line:

docker run -d --restart always --name logserver --net host \
    -v /etc/logserver/logserver.json:/logserver:json \
    stratoscale/logserver -addr :80

Configuration

Logserver is configured with a json configuration file. See example.

The json should be a dict with the following keys:

sources (list of source dicts): Logs sources, from which the logs are merged ans served.
parsers (list of parser dicts): Which parsers to apply to the log files.
global (dict of attributes): General configuration
cache (dict of attributes): Cache configuration
route (dict of attributes): Route configuration

Source Dict

name (string): Name of source, the name that this source will be shown as
url (URL string with supported schemes): URL of source.
open_tar (bool): Weather to treat tar files as directories, used for logs that are packed into a tar file.
open_journal (string): Open a journalctl directory as a log file. The value should be the journalctl directory from the source root.

Supported URL Schemes

Logserver supports different type of log sources, each has a different scheme:

file:// (URL string): Address in local file system. The file location can be absolute with file:///var/log or relative to the directory from which the command line was executed: file://./log.
sftp:// (URL string): Address of sftp server (or SSH server). The URL can contain a user name and password and path from the system root. For example: sftp://user:password@example.com:22/var/log
ssh:// (URL string): Address of ssh server. Obey the same rules as sftp server.
nginx+http://, nginx+https:// (URL string): Address of an nginx configured to serve files with autoindex on; directive. It supports both HTML and JSON autoindex_format.

Parser Dict

Logserver can parse each log line according to a list of configured parsers. Parsers can handle json logs (each line is a json dict), or regular logs with regular expression rules.

Each parser can be defined with the following keys:

glob (string): File pattern to apply this parser on.
time_formats (list of strings): Parse timestamp string according to those time formats. The given format should be in Go style time formats, or unix_int or unix_float.
json_mapping (dict): Parse each log line as a json, and map keys from that json to the UI expected keys. The keys are values that the UI expect, the values are keys from the file json.
regexp (Go style regular expression string): Parse each line in the long with this regular expression. the given regular expression should have named groups with the keys that the UI expects.
append_args (bool): (for json log) Add to msg all remaining json keys in format: key=value.

UI Keys

The UI expects the following keys in each parsed log:

msg: Log message.
time: Time stamp of log.
level: Log level.
args: If args are given, they will be injected into the log msg. Args value can be []interface{} Or map[string]interface{}, According to the log message.

Global Dict

content_batch_size
content_batch_time
search_max_size

Cache Dict

size
expiration

Route Dict

base_path
root_path

Stratoscale / logserver