We are using to host our packages!
CarbonAPI supports a significant subset of graphite functions [see COMPATIBILITY]. In our testing it has shown to be 5x-10x faster than requesting data from graphite-web.
For requirements see Requirements section below.
At this moment we are building packages for CentOS 7, Rockylinux 8 (should be compatible with RHEL 8), Debian 10, Debian 11, Debian 12 (testing), Ubuntu 18.04, Ubuntu 20.04, Ubuntu 22.04. Installation guides are available on packagecloud (see the links below).
Stable versions: Stable repo
Autobuilds (master, might be unstable): Autobuild repo
Configuration guides: docs/configuration.md and example config.
There are multiple example configurations available for different backends: prometheus, graphite-clickhouse, go-carbon, victoriametrics, IRONdb
Carbonapi can be configured by environment variables or by config file. For an example see carbonapi.example.yaml
$ ./carbonapi -config /etc/carbonapi.yaml
Request metrics will be dumped to graphite if corresponding config options are set, or if the GRAPHITEHOST/GRAPHITEPORT environment variables are found.
Request data will be stored in memory (default) or in memcache.
Configuration is described in docs
Every parameter in config file are mapped to environment variable. I.E.
concurency: 20
cache:
# Type of caching. Valid: "mem", "memcache", "null"
type: "mem"
upstreams:
backends:
- "http://10.0.0.1:8080"
- "http://10.0.0.2:8080"
That config can be replaced by
CARBONAPI_CONCURENCY=20
CARBONAPI_CACHE_TYPE=mem
CARBONAPI_UPSTREAMS_BACKENDS="http://10.0.0.1:8080 http://10.0.0.2:8080"
You should be only aware of logging: because carbonapi support a list of logger, env variables will replace only first logger.
If you apply variable LOGGER_FILE=stdout
to config:
logger:
- logger: ""
file: "stderr"
level: "debug"
encoding: "console"
encodingTime: "iso8601"
encodingDuration: "seconds"
- logger: ""
file: "carbonapi.log"
level: "info"
encoding: "json"
it will be equal to config:
logger:
- logger: ""
file: "stdout" # Changed only here
level: "debug"
encoding: "console"
encodingTime: "iso8601"
encodingDuration: "seconds"
- logger: ""
file: "carbonapi.log" # Not changed
level: "info"
encoding: "json"
auto
- carbonapi will do it's best to determine backend's protocol. Currently it can identify onlycarbonapi_v2_pb
orcarbonapi_v3_pb
carbonapi_v2_pb
,pb
,pb3
,protobuf
- carbonapi <0.11 style protocol. Supported by go-carbon and graphite-clickhouse older or equal version v0.11.7carbonapi_v3_pb
- new carbonapi protocol, that supports passing metadata through. Supported by carbonzipper >=1.0.0.alpha.3, graphite-clickhouse newer then v0.12.0 and go-carbon newer then v0.13.0carbonapi_v3_grpc
- grpc version of new carbonapi protocol. Currently no known implementation exists.msgpack
- messagepack based protocol, used in graphite-web 1.1 and metrictank. It's still experimental and might contain bugs.prometheus
- prometheus HTTP APIvictoriametrics
- special version of prometheus backend to use with VictoriaMetrics.irondb
- supports reading Graphite-compatible metrics from IRONdb from Circonus.
Golang compatibility matrix:
Golang Version | Last supported carbonapi version |
---|---|
1.10 | 0.10.0.1 |
1.12 | 0.12.6 |
1.16 / 1.17 | 0.15.6 |
1.18 | current |
Overall rule of thumb is that carbonapi supports last 2 major go versions. E.x. at this moment Go 1.19 and 1.18 are supported.
You can verify current versions that are being tested in CI Configuration.
CarbonAPI uses protobuf-based protocol to talk with underlying storages. For current version the compatibility list is:
- go-carbon >= 0.9.0 (Note: you need to enable carbonserver in go-carbon). Recommended to run latest version, that currently supports
carbonapi_v3_pb
- graphite-clickhouse any. That's alternative storage that doesn't use Whisper.
- metrictank - supported via
msgpack
protocol. Support is not very well tested and might contain bugs. Use with cautions. Tags are not supported. - carbonapi >= 0.5. Note: starting from carbonapi 3596e9647611e1f833a911d663747271623ec003 (post 0.8) carbonapi can be used as a zipper's replacement
- carbonserver@master (Note: you should probably switch to go-carbon in that case).
- carbonzipper >= 0.50. Please note, carbonzipper functionality was merged to carbonapi and it's no longer needed to run separate zipper. Current version of carbonzipper can be build from
cmd/carbonzipper
Currently building is tested regularly on amd64 (automated) and arm64 (manual) only. However from time to time, riscv64 is also tested manually.
For OS support: Linux is the only OS that is well tested for production usage. Theoretically nothing prevents from running carbonapi on *BSD, however its not tested by developers (but bugs will be accepted and eventually fixed). Running on macos is supported for testing purposes but it is not tested for any production use case. Other platforms are not tested and not supported.
For any other OS or Architectures bugs won't be actively worked on, but PRs that fixes the OS and doesn't break any other supported platforms are more than welcome.
For backends that uses proper database (e.x. graphite-clickhouse
) you should set maxBatchSize: 0
in your config file for this backend group.
For other backends (e.x. go-carbon) you should set it to some reasonable value. It increases response speed, but the cost is increased memory consumption.
Tag support was only tested with graphite-clickhouse
, however it should work with any other database.
The internal metrics are configured inside the graphite subsection and sent to your destinated host on an specified interval. The metrics are:
cache_items - if caching is enabled, this metric will contain many metrics are stored in cache cache_size - configured query cache size in bytes request_cache_hits - how many requests were served from cache. (this is for requests to /render endpoint) request_cache_misses - how many requests were not in cache. (this is for requests to /render endpoint) request_cache_overhead_ns - how much time in ns it took to talk to cache (that is useful to assess if cache actually helps you in terms of latency) (this is for requests to /render endpoint) find_requests - requests server by endpoint /metrics/find requests - requests served by endpoint /render requests_in_XX_to_XX - request response times in percentiles timeouts - number of timeouts while fetching from backend backend_cache_hits - how many requests were not read from backend backend_cache_misses - how many requests were not found in the backend
Some additional steps may be needed to build carbonapi with cairo rendering on MacOSX.
Install cairo:
$ brew install Caskroom/cask/xquartz
$ brew install cairo --with-x11
This program was originally developed for Booking.com. With approval from Booking.com, the code was generalised and published as Open Source on github, for which the author would like to express his gratitude.
In summer 2018, Booking.com forked version 0.11 of carbonapi and continued development in their own repo: github.com/bookingcom/carbonapi.
This code is licensed under the BSD-2 license.