This is the router (or relay, or reverse-proxy) for Graphite. It routes incoming records according to the specified rules. The Nanotube is designed for high-load systems. It is used at Booking.com to route up to 400k incoming and 1.2M outgoing records/sec on a single box.
- Get this repository with
go get -u github.com/bookingcom/nanotube - Navigate to it and run
make - Run it with
./nanotube -config config/config.toml -clusters config/clusters.toml -rules config/rules.toml
Run the full setup including input stack nanotube and go-carbon with:
docker-compose up
You can feed in sample data with:
echo "test1.test 5 $(date +%s)" | nc -c localhost 2003
As many netcat implementations exist, a parameter may be needed to instruct it to close the socket once data is sent. Such param will usually be -q0, -c or -N. Refer to your implementation man page.
Get it back with:
seq 8080 8081 | xargs -I {port} curl "localhost:{port}/render/?target=test1.test&format=json&from=$(( $(date +%s) - 300 ))&until=$(date +%s)"
To test the second store (alone or in conjunction) change the metric path to test2.test or test1.test2.
The only supported protocol is line. The records should follow the structure:
path.path.path value datetime
Please refer to the sample configs in the config folder for examples and documentation.
Nanotube validates every record it receives. Validation checks performed are:
- does the record consist of three fields
- is value in a record a valid real number
- is time in the record a valid epoch
It is possible to turn on record normalization with the NormalizeRecords option in general configuration. It is on by default. Normalizations performed are mainly for backward compatibility with carbon-c-relay, namely:
- Double or more subsequent dots in path are converted to a single dot: a.b..c -> a.b.c
- Prefix and trailing dots in path are removed: .a.b.c. -> a.b.c
- Spaces in record are normalized to a single space character: a.b.c␣␣␣1.231234567 -> a.b.c␣1.23␣1234567
- Characters not in the set
[0-9a-zA-Z-_:#]are replaced with_. Tags have their own allowed chars but are not supported for now.
Tags are not supported for now. See bookingcom#4
Please refer to the design doc docs/design.md.
This program was originally developed for Booking.com. With approval from Booking.com, the code was generalized and published as Open Source on GitHub, for which the authors would like to express their gratitude.