avaziman / RedisTimeSeries

Time Series data structure for Redis

RedisTimeSeries

RedisTimeSeries is a time-series database (TSDB) module for Redis, by Redis.

RedisTimeSeries can hold multiple time series. Each time series is accessible via a single Redis key (similar to any other Redis data structure).

What is a Redis time series?

A Redis time series comprises:

• Raw samples: each raw sample is a {time tag, value} pair.

  • Time tags are measured in milliseconds since January 1st, 1970, at 00:00:00.

    Time tags can be specified by the client or filled automatically by the server.

  • 64-bit floating-point values.

  The intervals between time tags can be constant or variable.

  Raw samples can be reported in-order or out-of-order.

  Duplication policy for samples with identical time tags can be set: block/first/last/min/max/sum.

• An optional configurable retention period.

  Raw samples older than the retention period (relative to the raw sample with the highest time tag) are discarded.

• Series Metadata: a set of name-value pairs (e.g., room = 3; sensorType = ‘xyz’).

  RedisTimeSeries supports cross-time-series commands. One can, for example, aggregate data over all sensors in the same room or all sensors of the same type.

• Zero or more compactions.

  Compactions are an economical way to retain historical data.

  Each compaction is defined by:

  • A timeframe. E.g., 10 minutes
  • An aggregator: min, max, sum, avg, …
  • An optional retention period. E.g., 10 year

  For example, the following compaction: {10 minutes; avg; 10 years} will store the average of the raw values measured in each 10-minutes time frame - for 10 years.

Examples of time series

• Sensor data: e.g., temperatures or fan velocity for a server in a server farm
• Historical prices of a stock
• Number of vehicles passing through a given road (count per 1-minute timeframes)

Features

• High volume inserts, low latency reads
• Query by start time and end-time
• Aggregated queries (Min, Max, Avg, Sum, Range, Count, First, Last, STD.P, STD.S, Var.P, Var.S, twa) for any time bucket
• Configurable maximum retention period
• Compactions - automatically updated aggregated timeseries
• Secondary index - each time series has labels (name-value pairs) which will allows to query by labels

Using with other tools metrics tools

In the RedisTimeSeries organization you can find projects that help you integrate RedisTimeSeries with other tools, including:

1. Prometheus - read/write adapter to use RedisTimeSeries as backend db.
2. Grafana - using the Redis Data Source.
3. Telegraf
4. StatsD, Graphite exports using graphite protocol.

Memory model

A time series is a linked list of memory chunks. Each chunk has a predefined size of samples. Each sample is a tuple of the time and the value of 128 bits, 64 bits for the timestamp and 64 bits for the value.

Setup

You can either get RedisTimeSeries setup in a Docker container or on your own machine.

Docker

To quickly try out RedisTimeSeries, launch an instance using docker:

docker run -p 6379:6379 -it --rm redis/redis-stack-server:latest

Build and Run it yourself

You can also build and run RedisTimeSeries on your own machine.

Major Linux distributions as well as macOS are supported.

Requirements

First, clone the RedisTimeSeries repository from git:

git clone --recursive https://github.com/RedisTimeSeries/RedisTimeSeries.git

Then, to install required build artifacts, invoke the following:

cd RedisTimeSeries
make setup

Or you can install required dependencies manually listed in system-setup.py.

If make is not yet available, the following commands are equivalent:

./deps/readies/bin/getpy3
./system-setup.py

Note that system-setup.py will install various packages on your system using the native package manager and pip. This requires root permissions (i.e. sudo) on Linux.

If you prefer to avoid that, you can:

• Review system-setup.py and install packages manually,
• Utilize a Python virtual environment,
• Use Docker with the --volume option to create an isolated build environment.

Build

make build

Binary artifacts are placed under the bin directory.

Run

In your redis-server run: loadmodule bin/redistimeseries.so

For more information about modules, go to the Redis official documentation.

Give it a try

After you setup RedisTimeSeries, you can interact with it using redis-cli.

Here we'll create a time series representing sensor temperature measurements. After you create the time series, you can send temperature measurements. Then you can query the data for a time range on some aggregation rule.

With redis-cli

$ redis-cli
127.0.0.1:6379> TS.CREATE temperature:3:11 RETENTION 60 LABELS sensor_id 2 area_id 32
OK
127.0.0.1:6379> TS.ADD temperature:3:11 1548149181 30
OK
127.0.0.1:6379> TS.ADD temperature:3:11 1548149191 42
OK
127.0.0.1:6379>  TS.RANGE temperature:3:11 1548149180 1548149210 AGGREGATION avg 5
1) 1) (integer) 1548149180
   2) "30"
2) 1) (integer) 1548149190
   2) "42"

Client libraries

Some languages have client libraries that provide support for RedisTimeSeries commands:

Project	Language	License	Author	Stars	Comment
Jedis	Java	MIT	Redis
JRedisTimeSeries	Java	BSD-3	Redis		Deprecated
redis-modules-java	Java	Apache-2	dengliming
redistimeseries-go	Go	Apache-2	Redis
rueidis	Go	Apache-2	Rueian
redis-py	Python	MIT	Redis
NRedisTimeSeries	.NET	BSD-3	Redis
phpRedisTimeSeries	PHP	MIT	Alessandro Balasco
redis-time-series	JavaScript	MIT	Rafa Campoy
redistimeseries-js	JavaScript	MIT	Milos Nikolovski
redis-modules-sdk	Typescript	BSD-3-Clause	Dani Tseitlin
redis_ts	Rust	BSD-3	Thomas Profelt
redistimeseries	Ruby	MIT	Eaden McKee
redis-time-series	Ruby	MIT	Matt Duszynski

Tests

The module includes a basic set of unit tests and integration tests.

Unit tests

To run all unit tests, follow these steps:

$ make unit_tests

Integration tests

Integration tests are based on RLTest, and specific setup parameters can be provided to configure tests. By default the tests will be ran for all common commands, and with variation of persistency and replication.

To run all integration tests in a Python virtualenv, follow these steps:

$ mkdir -p .env
$ virtualenv .env
$ source .env/bin/activate
$ pip install -r tests/flow/requirements.txt
$ make test

To understand what test options are available simply run:

$ make help

For example, to run the tests strictly desigined for TS.ADD command, follow these steps:

$ make test TEST=test_ts_add.py

Documentation

Read the docs at http://redistimeseries.io

Mailing List / Forum

Got questions? Feel free to ask at the RedisTimeSeries forum.

License

RedisTimeSeries is licensed under the Redis Source Available License 2.0 (RSALv2) or the Server Side Public License v1 (SSPLv1).