xiaof-github / logkit

Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.

logkit-community

中文版

Introduce

Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.

logkit-community Detail doc can be referred toWIKI

Support sources

• File: read data in file, including csv file，kafka-rest log，nginx log.
• Elasticsearch: read data in ElasticSearch.
• MongoDB: read data in MongoDB.
• MySQL: read data in MySQL.
• MicroSoft SQL Server: read data in Microsoft SQL Server.
• Postgre SQL: read data in PostgreSQL.
• Kafka: read data in Kafka.
• Redis: read data in Redis.
• Socket: read data via tcp\udp\unixsocket protocol.
• Http: reveive data in post request as http server.
• Script: support script and read data from the result.
• Snmp: auto read data from Snmp service.

Working method

logkit-community support multiple sources and can send kinds of data to Pandora, every data source relevant to a logic runner,a runner's workaround as follows:

Contributing

Weclome to contribute to logkit:

• fix or report bug
• pull issue improve wiki doc
• review code or create issues
• contribute code (contribute kinds of modules including reader、parser、sender and transformer）

Download

lastest stable：Go to Download page

History：Go to Releases

Trial：construct lastest logkit trial version every 5:00am (only for Linux 64 and Docker), you can download it (note: not include update of frontend).

• Linux 64 download

• Docker image nightly: docker pull wonderflow/logkit:nightly

Install and Usage

1. Download&Decompress logkit-community tool

• Linux

export LOGKIT_VERSION=<version number>
wget https://pandora-dl.qiniu.com/logkit_${LOGKIT_VERSION}.tar.gz && tar xvf logkit_${LOGKIT_VERSION}.tar.gz && rm logkit_${LOGKIT_VERSION}.tar.gz && cd _package_linux64/

• MacOS

export LOGKIT_VERSION=<version number>
wget https://pandora-dl.qiniu.com/logkit_mac_${LOGKIT_VERSION}.tar.gz && tar xvf logkit_mac_${LOGKIT_VERSION}.tar.gz && rm logkit_mac_${LOGKIT_VERSION}.tar.gz && cd _package_mac/

• Windows

please download https://pandora-dl.qiniu.com/logkit_windows_<LOGKIT_VERSION>.zip 并解压缩，go to directory

2. change logkit-community configuration

logkit.conf is logkit-community tool's configuration，mainly for specifing running resource and paths of runners.

Open logkit.conf, for example：

{
    "max_procs": 8,
    "debug_level": 1,
    "clean_self_log":true,
    "bind_host":"localhost:3000",
    "static_root_path":"./public",
    "confs_path": ["confs*"]
}

For simply use, you can only focus on three options:

1. bind_host port of logkit we。
2. static_root_path statistic resource path of logkit page, recommand to use absolute path note：old version moved to "public-old" directory。
3. confs_path including add conf in web, logkit also support monitor directory to add runners. (if you only need to add logkit runner in web, you can ignore this option)

3. startup logkit-community tool

./logkit -f logkit.conf

4. Open logkit-community config page in web

the web url is the value of bind_host configured in step 2

Contribute frontend code

refer to README file：logkitweb/README.md

Install and startup from source code

go build -o logkit logkit.go
./logkit -f logkit.conf

startup logkit using docker

docker pull wonderflow/logkit:<version>
docker run -d -p 3000:3000 -v /local/logkit/dataconf:/app/confs -v /local/log/path:/logs/path logkit:<version>

Deploying logkit in Kubernetes

get configs deploying in Kubernetes

curl -L -O https://raw.githubusercontent.com/qiniu/logkit/master/deploy/logkit_on_k8s.yaml

enjoy it！