rsync metrics exporter for Prmoetheus.io
- linux OS with tail + rsync installed
- Golang >= 1.12
- [OPTIONAL] Docker
exporter.go
=> main codeexporter_test.go
=> unit tests
This utility is written in GO and uses tail -f -n +1 [logfile]
STDOUTPUT to parse a live logfile.
The overall logic is:
tail
is invoked and its output is piped usingStdoutPipe()
- A
bufio.Scanner
is attached to the pipe. It provides a convenient interface for reading data such as a file of newline-delimited lines of text. Created withbufio.NewScanner
which by default assumes aScanLines
split function. - A GO subroutine runs a WHILE-type loop that scans every new line in the file.
- The line is passed to a log parser that extracts meaningful data for the metrics using carefully tailored
strings.Split
functions - Extracted values are passed to the relevant metric object using function
recordMetrics(metricType string, value int)
- Metrics are exposed using
promhttp
handler - For an explanation on metric choices, go to Metrics section
- Metrics for
Prometheus.io
are exposed to HTTP/metrics
endpoint onPORT 2112
. To reach locally just issuewget http://localhost:2112/metrics
- Default log is
./rsync_example.log
The following metrics are exposed to Prometheus.io
- "connections_to_rsync" => The total number of connections to rsync daemon
- "rsync_executions" => The total number of rsync executions
- "data_sent" => The total data sent (bytes)
- "data_received" => The total data received (bytes)
All 4 metrics are COUNTER
types as they can only increase and can be easily transformed in rates using rate()
function in Prometheus to get interesting derived metrics, like connections/min or send-receive bytes/sec.
Once the repository is in a <folder>
:
-
cd <folder>
-
Launch the
run.sh [full path of rsync logfile to parse]
shell script with your shell. For example, using sh:sh run.sh [full path of rsync logfile to parse]
Once the repository is in a <folder>
:
cd <folder>
go build -o exporter
(to avoid that the executable has the default naming<folder>
)exporter [PATH_TO_RSYNC_LOGFILE]
to start monitoring the rsync logfile
There is a Dockerfile to build an Alpine based container with the compiled executable (called app
).
Once the repository is in a <folder>
:
cd <folder>
docker build --rm -f "Dockerfile" -t <AN IMAGE NAME HERE>:latest .
docker run --rm -it -p 2112:2112 -v <PATH TO RSYNC LOG FILE IN HOST>:<MOUNT POINT IN THE CONTAINER>:ro <AN IMAGE NAME HERE>:latest sh
- Run the app with:
app [MOUNT POINT IN THE CONTAINER]
- Metrics will be exposed to
http://localhost:2112/metrics
or, if you usedocker-machine
,http://DOCKER_MACHINE_IP:2112/metrics
Once the app is compiled and named exporter
:
exporter [OPTIONS] PATH_TO_RSYNC_LOGFILE
--help, -h
Help: displays all available options
-p
TCP port to listen fo /metrics endpoint
Some unit tests are provided.
To run tests:
go test
IMPORTANT NOTICE IF USING DOCKER IMAGE: Tests require gcc
, so the provided Alpine based container is not a suitable option to execute them. Use the standard golang:1.12
instead or run them in a linux environment with gcc
and Go 1.12
. For convenience there is a Dockerfile for this purpose.
To use it:
docker build --rm -f "Dockerfile4test" -t <AN IMAGE NAME SPECIFIC FOR TEST HERE>:latest .
This package is released under the GPLv3 license