shanipribadi / skyhook

Skyhook is a Redis API compatible gateway to Aerospike Database

Skyhook

Skyhook is a Redis API compatible gateway to Aerospike Database.

• Overview
• Connecting a Redis Client to Skyhook
• Redis Command Coverage
• Known Limitations
• Deploying Skyhook
  • Prerequisites
  • Installing
    • Building from Source
  • Running the Server
    • Configuring Skyhook
• License

Overview

Skyhook is designed as a standalone server application written in Kotlin, which accepts Redis protocol commands and projects them to an Aerospike cluster using the Aerospike Java client under the hood. It uses Netty as a non-blocking I/O client-server framework.

This project is now in beta. If you're an enterprise customer feel free to reach out to our support with feedback and feature requests. We appreciate feedback from the Aerospike community on issues related to Skyhook.

Connectivity

Any Redis client can connect to Skyhook as if it were a regular Redis server.

For tests purposes use redis-cli or even the nc (or netcat) utility:

echo "GET key1\r\n" | nc localhost 6379

Redis Command Coverage

List of the supported Redis commands

Operation	Description
APPEND key value	If key already exists and is a string, this command appends the value at the end of the string. If key does not exist it is created and set as an empty string.
AUTH [username] password	The AUTH command authenticates the current connection.
BGSAVE	Returns OK.
COMMAND	Returns Array reply of details about all Redis commands.
COMMAND COUNT	Returns Integer reply of number of total commands in this Redis server.
COMMAND INFO command-name [command-name ...]	Returns Array reply of details about multiple Redis commands.
DBSIZE	Returns the number of keys in the currently-selected database.
DECR key	Decrements the number stored at key by one.
DECRBY key decrement	Decrements the number stored at key by decrement.
DEL key	Removes the specified key.
DISCARD	Flushes all previously queued commands in a transaction and restores the connection state to normal.
ECHO message	Returns message.
EXEC	Executes all previously queued commands in a transaction and restores the connection state to normal.
EXISTS key [key ...]	Returns if key exists.
EXPIRE key seconds	Set a timeout on key. After the timeout has expired, the key will automatically be deleted.
EXPIREAT key timestamp	EXPIREAT has the same effect and semantic as EXPIRE, but instead of specifying the number of seconds representing the TTL (time to live), it takes an absolute Unix timestamp (seconds since January 1, 1970).
FLUSHALL	Delete all the keys of all the existing databases, not just the currently selected one.
FLUSHDB	Delete all the keys of the currently selected DB.
GETSET key value	Atomically sets key to value and returns the old value stored at key.
GET key	Get the value of key.
HDEL key field [field ...]	Removes the specified fields from the hash stored at key.
HEXISTS key field	Returns if field is an existing field in the hash stored at key.
HGETALL key	Returns all fields and values of the hash stored at key.
HGET key field	Returns the value associated with field in the hash stored at key.
HINCRBYFLOAT key field increment	Increment the specified field of a hash stored at key, and representing a floating point number, by the specified increment.
HINCRBY key field increment	Increments the number stored at field in the hash stored at key by increment.
HKEYS key	Returns all field names in the hash stored at key.
HLEN key	Returns the number of fields contained in the hash stored at key.
HMGET key field [field ...]	Returns the values associated with the specified fields in the hash stored at key.
HMSET key field value [field value ...]	Sets the specified fields to their respective values in the hash stored at key.
HSCAN key cursor [MATCH pattern] [COUNT count]	See SCAN for HSCAN documentation.
HSETNX key field value	Sets field in the hash stored at key to value, only if field does not yet exist.
HSET key field value [field value ...]	Sets field in the hash stored at key to value.
HSTRLEN key field	Returns the string length of the value associated with field in the hash stored at key.
HVALS key	Returns all values in the hash stored at key.
INCRBYFLOAT key increment	Increment the string representing a floating point number stored at key by the specified increment.
INCRBY key increment	Increments the number stored at key by increment.
INCR key	Increments the number stored at key by one.
LINDEX key index	Returns the element at index index in the list stored at key.
LLEN key	Returns the length of the list stored at key.
LOLWUT [VERSION version]	The LOLWUT command displays the Redis version.
LPOP key [count]	Removes and returns the first elements of the list stored at key.
LPUSHX key element [element ...]	Inserts specified values at the head of the list stored at key, only if key already exists and holds a list.
LPUSH key element [element ...]	Insert all the specified values at the head of the list stored at key.
LRANGE key start stop	Returns the specified elements of the list stored at key.
MGET key [key ...]	Returns the values of all specified keys.
MSET key value [key value ...]	Sets the given keys to their respective values.
MSETNX key value [key value ...]	Sets the given keys to their respective values. MSETNX will not perform any operation at all even if just a single key already exists.
MULTI	Marks the start of a transaction block. Subsequent commands will be queued for execution using EXEC.
PERSIST key	Remove the existing timeout on key, turning the key from volatile (a key with an expire set) to persistent.
PEXPIRE key milliseconds	This command works exactly like EXPIRE but the time to live of the key is specified in milliseconds instead of seconds.
PEXPIREAT key milliseconds-timestamp	PEXPIREAT has the same effect and semantic as EXPIREAT, but the Unix time at which the key will expire is specified in milliseconds instead of seconds.
PING [message]	Returns PONG if no argument is provided, otherwise return a copy of the argument as a bulk.
PSETEX key milliseconds value	PSETEX works exactly like SETEX with the sole difference that the expire time is specified in milliseconds instead of seconds.
PTTL key	Returns the amount of remaining time in milliseconds.
QUIT	Returns OK.
RANDOMKEY	Return a random key from the currently selected database.
RESET	Returns 'RESET'.
RPOP key [count]	Removes and returns the last elements of the list stored at key.
RPUSHX key element [element ...]	Inserts specified values at the tail of the list stored at key, only if key already exists and holds a list.
RPUSH key element [element ...]	Insert all the specified values at the tail of the list stored at key.
SADD key member [member ...]	Add the specified members to the set stored at key.
SAVE	Returns OK.
SCAN cursor [MATCH pattern] [COUNT count] [TYPE type]	The SCAN command and the closely related commands SSCAN, HSCAN and ZSCAN are used in order to incrementally iterate over a collection of elements.
SCARD key	Returns the set cardinality (number of elements) of the set stored at key.
SETEX key seconds value	Set key to hold the string value and set key to timeout after a given number of seconds.
SETNX key value	Set key to hold string value if key does not exist.
SET key value	Set key to hold the string value. If key already holds a value, it is overwritten, regardless of its type.
SINTER key [key ...]	Returns the members of the set resulting from the intersection of all the given sets.
SINTERSTORE destination key [key ...]	This command is equal to SINTER, but instead of returning the resulting set, it is stored in destination.
SISMEMBER key member	Returns if member is a member of the set stored at key.
SMEMBERS key	Returns all the members of the set value stored at key.
SREM key member [member ...]	Remove the specified members from the set stored at key.
SSCAN key cursor [MATCH pattern] [COUNT count]	See SCAN for SSCAN documentation.
STRLEN key	Returns the length of the string value stored at key. An error is returned when key holds a non-string value.
SUNION key [key ...]	Returns the members of the set resulting from the union of all the given sets.
SUNIONSTORE destination key [key ...]	This command is equal to SUNION, but instead of returning the resulting set, it is stored in destination.
TIME	Returns the current server time.
TOUCH key [key ...]	Alters the last access time of a key(s). A key is ignored if it does not exist.
TTL key	Returns the remaining time to live of a key that has a timeout.
TYPE key	Returns the string representation of the type of the value stored at key. The different types that can be returned are: string, list, set, zset, hash and stream.
UNLINK key [key ...]	This command is an alias to DEL.
ZADD key [NX/XX] [GT/LT] [CH] [INCR] score member [score member ...]	Adds all the specified members with the specified scores to the sorted set stored at key.
ZCARD key	Returns the sorted set cardinality (number of elements) of the sorted set stored at key.
ZCOUNT key min max	Returns the number of elements in the sorted set at key with a score between min and max.
ZINCRBY key increment member	Increments the score of member in the sorted set stored at key by increment.
ZLEXCOUNT key min max	When all the elements in a sorted set are inserted with the same score, in order to force lexicographical ordering, this command returns the number of elements in the sorted set at key with a value between min and max.
ZMSCORE key member [member ...]	Returns the scores associated with the specified members in the sorted set stored at key.
ZPOPMAX key [count]	Removes and returns up to count members with the highest scores in the sorted set stored at key.
ZPOPMIN key [count]	Removes and returns up to count members with the lowest scores in the sorted set stored at key.
ZRANDMEMBER key [count [WITHSCORES]]	When called with just the key argument, return a random element from the sorted set value stored at key.
ZRANGE key min max [BYSCORE/BYLEX] [REV] [LIMIT offset count] [WITHSCORES]	Returns the specified range of elements in the sorted set stored at .
ZRANGEBYLEX key min max [LIMIT offset count]	When all the elements in a sorted set are inserted with the same score, in order to force lexicographical ordering, this command returns all the elements in the sorted set at key with a value between min and max.
ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]	Returns all the elements in the sorted set at key with a score between min and max (including elements with score equal to min or max). The elements are considered to be ordered from low to high scores.
ZRANGESTORE dst src min max [BYSCORE/BYLEX] [REV] [LIMIT offset count]	This command is like ZRANGE, but stores the result in the destination key.
ZRANK key member	Returns the rank of member in the sorted set stored at key, with the scores ordered from low to high.
ZREM key member [member ...]	Removes the specified members from the sorted set stored at key.
ZREMRANGEBYLEX key min max	When all the elements in a sorted set are inserted with the same score, in order to force lexicographical ordering, this command removes all elements in the sorted set stored at key between the lexicographical range specified by min and max.
ZREMRANGEBYRANK key start stop	Removes all elements in the sorted set stored at key with rank between start and stop.
ZREMRANGEBYSCORE key min max	Removes all elements in the sorted set stored at key with a score between min and max (inclusive).
ZREVRANGE key start stop [WITHSCORES]	Returns the specified range of elements in the sorted set stored at key.
ZREVRANGEBYLEX key max min [LIMIT offset count]	Apart from the reversed ordering, ZREVRANGEBYLEX is similar to ZRANGEBYLEX.
ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]	Returns all the elements in the sorted set at key with a score between max and min (including elements with score equal to max or min). In contrary to the default ordering of sorted sets, for this command the elements are considered to be ordered from high to low scores.
ZSCAN key cursor [MATCH pattern] [COUNT count]	See SCAN for ZSCAN documentation.

Known Limitations

• A partial but growing list of Redis commands. See Redis Command Coverage.
• By default, an Aerospike namespace does not allow for TTLs. Read more on how to set up expiration and eviction support.
• Like Redis Cluster, Skyhook supports a single Redis 'database 0', which maps to a single namespace and set in the Aerospike Database.
• Will not try to implement the cluster operations sub-commands of CLUSTER, CLIENT, CONFIG, MEMORY, MONITOR, LATENCY.
• No support for Pub/Sub commands.
• No support for Lua scripts.

Deploying Skyhook

Prerequisites

• Java 8 or later
• Aerospike Server version 4.9+

Installing

Building from Source

To build the project:

./gradlew clean build

A fat executable jar will be created under the build/libs folder.

Running

Usage:

% java -jar skyhook-[version]-all.jar -h

Usage: skyhook [-h] [-f=<configFile>]
Redis to Aerospike proxy server
  -f, --config-file=<configFile>
               yaml formatted configuration file
  -h, --help   display this help and exit

To run the server:

java -jar skyhook-[version]-all.jar -f config/server.yml

The configuration file carries all the settings the server needs and is in YAML format. An example configuration file can be found in the config folder. If no configuration file is specified, the default settings will be applied.

[main] INFO  c.a.skyhook.SkyhookServer$Companion - Starting the Server...

Now the server is listening to the config.redisPort (default: 6379) and is ready to serve.

If you wish to deploy Skyhook as a cluster of nodes, you can find some example configurations here.

Running on Docker

Build an image:

docker build -t skyhook .

Run as a Docker container:

docker run -d --name=skyhook -p 6379:6379 skyhook 

The image uses the repository configuration file by default. Bind mount a custom file to configure the server:

docker run -d --name=skyhook -v "$(pwd)"/config/server.yml:/app/server.yml -p 6379:6379 skyhook

Configuration Properties

Property name	Description	Default value
hostList	The host list to seed the Aerospike cluster.	localhost:3000
namespase	The Aerospike namespace.	test
set	The Aerospike set name.	redis
clientPolicy	The Aerospike Java client ClientPolicy configuration properties.	ClientPolicyConfig
bin	The Aerospike value bin name.	b
typeBin	The Aerospike value type bin name.	t
redisPort	The server port to bind to.	6379
workerThreads1	The Netty worker group size.	number of available cores
bossThreads	The Netty acceptor group size.	2

¹ Used to configure the size of the Aerospike Java Client EventLoops as well.

License

Licensed under an Apache 2.0 License.

This is an active open source project. You can contribute to it by trying Skyhook, providing feedback, reporting bugs, and implementing more Redis commands.