Pika
Introduction中文
Pika is a persistent huge storage service , compatible with the vast majority of redis interfaces (details), including string, hash, list, zset, set and management interfaces. With the huge amount of data stored, redis may suffer for a capacity bottleneck, and pika was born for solving it. Except huge storage capacity, pika also support master-slave mode by slaveof command, including full and partial synchronization. You can also use pika together with twemproxy or codis(pika has supported data migration in codis，thanks left2right) for distributed Redis solution
UserList
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Feature
- huge storage capacity
- compatible with redis interface, you can migrate to pika easily
- support master-slave mode (slaveof)
- various management interfaces
For developer
Releases
The User can download the binary release from releases or compile the source release.
Dependencies
- snappy - a library for fast data compression
- glog - google log library
Upgrade your gcc to version at least 4.8 to get C++11 support.
Supported platforms
-
linux - Centos 5&6
-
linux - Ubuntu
If it comes to some missing libs, install them according to the prompts and retry it.
Compile
Upgrade your gcc to version at least 4.8 to get C++11 support.
Get the source code
git clone https://github.com/Qihoo360/pika.git
Then compile pika, all submodules will be updated automatically.
make
Usage
./output/bin/pika -c ./conf/pika.conf
Performance (provided by deep011)
test environment
CPU module：Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
CPU threads：56
MEMORY：256G
DISK：3T flash
NETWORK：10GBase-T/Full * 2
OS：centos 6.6
benchmark tools
Test 1
Purpose
With different number of pika worker threads, we test pika's max QPS.
Condition
pika db_size : 800G
value : 128bytes
Threads are not bound to CPU cores.
Result
Description : Horizontal axis is the number of worker threads; Vertical axis is the QPS. The value size is 128bytes. set3/get7 means 30% set and 70% get.
Conclusion
The best pika worker threads number is 20-24.
Test 2
Purpose
With the optimal worker threads number, we test pika's round-trip time.
Condition
pika db_size：800G
value：128bytes
Result
====== GET ======
10000000 requests completed in 23.10 seconds
200 parallel clients
3 bytes payload
keep alive: 1
99.89% <= 1 milliseconds
100.00% <= 2 milliseconds
100.00% <= 3 milliseconds
100.00% <= 5 milliseconds
100.00% <= 6 milliseconds
100.00% <= 7 milliseconds
100.00% <= 7 milliseconds
432862.97 requests per second====== SET ======
10000000 requests completed in 36.15 seconds
200 parallel clients
3 bytes payload
keep alive: 1
91.97% <= 1 milliseconds
99.98% <= 2 milliseconds
99.98% <= 3 milliseconds
99.98% <= 4 milliseconds
99.98% <= 5 milliseconds
99.98% <= 6 milliseconds
99.98% <= 7 milliseconds
99.98% <= 9 milliseconds
99.98% <= 10 milliseconds
99.98% <= 11 milliseconds
99.98% <= 12 milliseconds
99.98% <= 13 milliseconds
99.98% <= 16 milliseconds
99.98% <= 18 milliseconds
99.99% <= 19 milliseconds
99.99% <= 23 milliseconds
99.99% <= 24 milliseconds
99.99% <= 25 milliseconds
99.99% <= 27 milliseconds
99.99% <= 28 milliseconds
99.99% <= 34 milliseconds
99.99% <= 37 milliseconds
99.99% <= 39 milliseconds
99.99% <= 40 milliseconds
99.99% <= 46 milliseconds
99.99% <= 48 milliseconds
99.99% <= 49 milliseconds
99.99% <= 50 milliseconds
99.99% <= 51 milliseconds
99.99% <= 52 milliseconds
99.99% <= 61 milliseconds
99.99% <= 63 milliseconds
99.99% <= 72 milliseconds
99.99% <= 73 milliseconds
99.99% <= 74 milliseconds
99.99% <= 76 milliseconds
99.99% <= 83 milliseconds
99.99% <= 84 milliseconds
99.99% <= 88 milliseconds
99.99% <= 89 milliseconds
99.99% <= 133 milliseconds
99.99% <= 134 milliseconds
99.99% <= 146 milliseconds
99.99% <= 147 milliseconds
100.00% <= 203 milliseconds
100.00% <= 204 milliseconds
100.00% <= 208 milliseconds
100.00% <= 217 milliseconds
100.00% <= 218 milliseconds
100.00% <= 219 milliseconds
100.00% <= 220 milliseconds
100.00% <= 229 milliseconds
100.00% <= 229 milliseconds
276617.50 requests per secondConclusion
Both the 99.9% get/set RTT are below 2ms.
Test 3
Purpose
With the optimal worker threads number, we test the max qps of different commands.
Condition
pika worker threads：20
key count：10000
field count：100（except list）
value：128bytes
commands execute times：10000000(except lrange)
Result
PING_INLINE: 548606.50 requests per second
PING_BULK: 544573.31 requests per second
SET: 231830.31 requests per second
GET: 512163.91 requests per second
INCR: 230861.56 requests per second
MSET (10 keys): 94991.12 requests per second
LPUSH: 196093.81 requests per second
RPUSH: 195186.69 requests per second
LPOP: 131156.14 requests per second
RPOP: 152292.77 requests per second
LPUSH (needed to benchmark LRANGE): 196734.20 requests per second
LRANGE_10 (first 10 elements): 334448.16 requests per second
LRANGE_100 (first 100 elements): 50705.12 requests per second
LRANGE_300 (first 300 elements): 16745.16 requests per second
LRANGE_450 (first 450 elements): 6787.94 requests per second
LRANGE_600 (first 600 elements): 3170.38 requests per second
SADD: 160885.52 requests per second
SPOP: 128920.80 requests per second
HSET: 180209.41 requests per second
HINCRBY: 153364.81 requests per second
HINCRBYFLOAT: 141095.47 requests per second
HGET: 506791.00 requests per second
HMSET (10 fields): 27777.31 requests per second
HMGET (10 fields): 38998.52 requests per second
HGETALL: 109059.58 requests per second
ZADD: 120583.62 requests per second
ZREM: 161689.33 requests per second
PFADD: 6153.47 requests per second
PFCOUNT: 28312.57 requests per second
PFADD (needed to benchmark PFMERGE): 6166.37 requests per second
PFMERGE: 6007.09 requests per secondTest 4
Purpose
Compare the pika max qps with the redis.
Condition
pika worker threads：20
key count：10000
field count：100（except list）
value：128bytes
commands execute times：10000000(except lrange)
Redis version：3.2.0
Result
Documents
Contact Us
Mail: g-infra-bada@360.cn
QQ group: 294254078
For more information about Pika, Atlas and some other technology please pay attention to our Hulk platform official account
