Redis SSL support
antirez opened this issue · comments
HQ for ideas / proposals / code about Redis and SSL support.
I'll be looking over the code I wrote over Christmas... I have Dec 18th through Jan 2nd off, so I should have plenty of time finish testing 2.6, and get 2.7 and 2.8 updates merged in. I'll also see if I can create a new branch directly from yours and merge my changes into it. (I'll be working on things in that order probably). Should I start with your 2.6, 2.7 or 2.8?
Thanks @bbroerman30, ideally the best thing is to start from the "unstable" branch. Back porting will not be hard and I can take care of it, but for new stuff it's ideal to hack on the development tree.
I'll fork off of that then, and once I get 2.6 tested (and go through the code and remember what I did and why) I'll start looking at the unstable branch and making changes in there. I'll hold off on 2.7 and 2.8 updates on my side...
Just wanted to say thank you for opening this issue!
To anyone who might happen to ask "why not stunnel/stud/spiped/(insert proxy here)/etc" we operate nearly a hundred of these tunnels and have to manage the overhead of provisioning individual certificates, trust stores, adding new certificates to these trust stores for key rotation, ensuring that apps, tunnels, and Redis are started up and taken down in awareness of this fact, and have to implement key rotation within these constraints.
It's very easy to say "just use stunnel". If you have any actual experience maintaining a highly available, secure infrastructure using dozens or hundreds of stunnels, I think you'll quickly become aware the reality of the situation is: easier said than done.
Native SSL support would eliminate all sorts of operational overhead we have around encrypting all of our Redis connections.
Last but not least, I'll leave you the quotes reel from when I originally told my coworkers that there may be a solution on the horizon for native SSL support in Redis:
"This is great."
"YES DIE STUNNELS DIE"
"Nice!"
"DO THIS OH GOD PLEASE DO THIS"
"Very nice!"
"Yes yes yes."
These quotes come from people operationally responsible for maintaining our stunnel-based infrastructure. I want to make these people's lives easier.
Ditto the above, and doubly so for the client drivers. Server operators can (sort of) cope, but getting clients to use stunnel is a considerably bigger chore for all involved.
I'll happily take any implementation over none, but regardless here are some ideas I experimented with.
In a nutshell, I decided SCRAM-SHA1-PLUS with tls-server-end-point channel-binding would let me fix the most Redis issues at once with the least work. This delivers some compelling advantages when the TLS and Authentication mechanisms of an application can interact.
As for implementation, I wrote some prototype code for this (in Go, presuming it'd have to be a proxy anyway) and was getting some traction, but then had to go work on something else. If there's sufficient interest, I may revisit that decision. Regardless, here is why I made these choices:
- Detecting MITM is simplified.
- Doesn't break TLS concentrators.
- Avoids persistent breaks via challenge-response.
From first to last:
Detecting MITM is simplified
This is a feature of "channel-binding". In a nutshell, the pre-shared secret (password) is mixed with both the client and server's view of the TLS state (e.g. presented public key) on the connection. A mismatch (e.g. decrypt-and-re-encrypt with another key) will present as an authentication failure.
This is a very cool way to avoid the complexity of certificate pinning or CAs provided one continues to use the password/pre-shared-key model, as I suspect pre-shared-keys will remain very useful for a long time.
This is known by the lingo SCRAM-SHA1-PLUS. In my prototype, deviating from the standard, to only support this "PLUS" variant, and not the non-channel-bound version.
Doesn't break TLS concentrators
This has to do with the choice of tls-server-end-point.
In my prototype, deviating from the standard, I was going to skip the otherwise mandatory support for tls-unique, as with a proxy design my own program was itself akin to a TLS concentrator. What this amounts to is mixing in the fingerprint of the public key encrypting the channel into the HMACs passed around (see the "algorithm" section).
Avoids persistent breaks via challenge-response
If one manages to get ahold of a decrypted session, salted challenge-response avoids the long-lived password from being compromised. This is a common feature of older methods like DIGEST-MD5.
SCRAM has some useful properties, but the most useful is that it's simpler to implement than even venerable older methods. Otherwise interesting but not as important to me is that it supports holding securely derived keys on the server, so that one does not compromise the password if the server reads out its own passwd-ish file.
The algorithm
It is seen with explanation of the notation in the SCRAM rfc linked above, but here's a taste (and lets you find the section):
SaltedPassword := Hi(Normalize(password), salt, i)
ClientKey := HMAC(SaltedPassword, "Client Key")
StoredKey := H(ClientKey)
AuthMessage := client-first-message-bare + "," +
server-first-message + "," +
client-final-message-without-proof
ClientSignature := HMAC(StoredKey, AuthMessage)
ClientProof := ClientKey XOR ClientSignature
ServerKey := HMAC(SaltedPassword, "Server Key")
ServerSignature := HMAC(ServerKey, AuthMessage)
Status Update: I have a clone of antirez/redis and am working on merging in the changes from my 2.6.17 branch into the trunk. I started with a 3-way diff, and merged everything I could from that, and then started compiling and working out all the compile errors. I have the initial merge done, and checked into my clone. It compiles, and I can connect with redis-cli but there are still critical errors reported by the self test. I'll be working over the next week (while i'm on vacation) to try and sort these out. After my vacation is over, I'll still be able to work some evenings and weekends until it's all working...
@bbroerman30 Is this work already public?
@bbroerman30 great to hear!
Was able to generate a self-signed certificate, add it to the root store, connect a client to a server with SSL, verified that it would not connect without, verified it checked the certificate, connected a slave with SSL, and verified it checked the certificate and validated the configured common name... Also, ran the runtest-sentinel and runtest-cluster with minimal errors without SSL (the errors I had were probably due to the tiny box I'm running this on... it's a Athlon XP 2600 with 1G ram) I will run them with SSL over the next couple days.
@badboy It's in my fork of redis (in the trunk). I checked in the changes last night to that repo. I have also been testing my sslredis 2.6.17 to make sure it's working properly as well... I'm using a copy of antirez's trunk for baselines on the unstable, and my sslredis 2.6.17 as the baseline for SSL...
Thanks for your work.
still have work to do... unstable branch is having issues on initial replication to slave. 2.6.17 was working perfectly, so I'm trying to figure out the differences.
ok, got initial replication happening (simple case, full dump) and continual replication between a master and 1 slave. Looking through the code, I think after I get it all working, I want to refactor... creating a structure that will replace the socket file descriptor. The structure should contain the fd, the anetSSLConnection, and a flag that indicates which to look at. Each place the fd (client.fd, server.fd, server.repl_transfer_s, etc.) is used it will be changed to this structure. I'm also thinking of a helper function that encapsulates SSL_write and write, as well we SSL_read and read. This new function will take the new structure.
Having issues getting a cluster running... Same error with the standard (unmodififed) unstable version as with my SSL version. All of the instances of redis server crash with a core dump when I tell the ./redis-trib.rb script to save the configuration. I'm running on a small Athlon XP 2600 box with 1Gb of ram, running stock Ubuntu 14.10 and following the instructions from the Redis cluster tutorial. I only have a couple days left of vacation, and would like to get this initial code change validated.
I don't really want to get into a refactor without knowing that the existing code is working.
@bbroerman30 Can you figure it out yourself or is there anything we can help with?
I'm pretty sure it's related to the small size of the linux box I have available... Do you have a box that has successfully run a small cluster before? If so, I would appreciate someone helping with running a simple cluster test
I've got some DigitalOcean credit left and access to other virtual servers. Just tell me what to do and I spin up some instances.
Can you give it a quick try with regular redis unstable? The instructions i was following are here: http://redis.io/topics/cluster-tutorial
Will do in a moment
Created the cluster with redis-trib. Tested with redis-cli -c. Everything seems to work fine so far.
P.S.: I'm also available on IRC (freenode)
@badboy Awesome. Can you try with the ssl version? Without SSL enabled. You can get it from my repo at https://github.com/bbroerman30/redis. I'm in and out today as it's the last day of my vacation, and I'm trying to get a bunch of other things done before I go back to the normal grind. I really appreciate your help!!!
Any hints for setting up the ssl cert and key?
Update: Sorry, over-read that I should test without SSL, even easier then.
yeah. Setting SSL isn't too bad, except that the Ruby and Tcl clients aren't set up for SSL. I created my own self-signed private key and cert, then installed the cert into the trusted root store on the box. Setting up the trusted root store depends on the OS...
After that, I set up the redis.conf like:
ssl true # turn on SSL (makes all connections require ssl)
ssl_ca_root_dir /usr/share/ca-certificates # The trusted root store for openSSL.
ssl_cert_file /home/brad/redis/cacert.pem # The self-signed cert the server is running with.
ssl_pk_file /home/brad/redis/privkey.pem # The private key I created the cert with.
ssl_cert_pass Pidb95 # Password for the private key/cert.
ssl_dhk_file /home/brad/redis/dHParam.pem # Diffie-Hellman key file .
ssl_cert_common_name bbroerman.net # This is used by the slaves to validate the CN in the cert (as most slaves connect by IP and not name. This is optional)
Ok, got it compiled and running. 6 instances (3 master, 3 slave), but for some reason I can't create a cluster, it hangs on CLUSTER MEET. (but no crash)
Ok, it does crash. Upstart was just fast enough to respawn the process without me noticing.
Log output: https://gist.github.com/badboy/9e16cf070de82c9309be
For some reason this gets called even though I did not enable ssl:
https://github.com/bbroerman30/redis/blob/unstable/src/cluster.c#L1928
That error should be fixed now. I got a bit more picky (as i should have been initially) on the initial setting of link->ssl's attributes on createClusterLink() and setting it's properties when an SSL connection is made. I wish I could run it over here (even the original unmodified unstable won't run cluster on my box)
It works now as expected. redis-trib can create a cluster and I can set keys as expected.
Your local machine must be really underpowered if it doesn't work. I could give you a small virtual machine to test on if you want.
Thanks. I may have time tomorrow to run through tests. The box I have here is one of the kids old computers. Athlon XP 2600 with 1G ram, running Ubuntu 14.10 32 bit.
Ok, just drop me a note if you want a box to test on.
Thanks! I'm running through make test currently with my latest changes, with SSL on. I had 1 error with it off (got the same error on the original version, so I'm not worried) Then I am going to try runtest-sentinel and runtest-cluster, and see what I get. In the morning, after my other stuff, I'll see if I can get a cluster going here one more time. If I can't I'll drop you a note.
Do you (or anyone here) know where this functionality resides? The failed test on normal mode is:
Can't detect write load from background clients.
It worked with SSL turned on, so I imagine I have a typo somewhere.
I'm assuming runtest-sentinel and runtest-cluster don't use the main redis.conf in the root directory (and therefore don't use SSL)... Everything is working there. I'm going to look over the tcl scripts to see if I can get them to do SSL
None of the tests use the redis.conf in the main directory.
The tests create the config on-the-fly, see https://github.com/bbroerman30/redis/blob/unstable/tests/cluster/run.tcl#L14-L17
Please save yourself and us some time and try to follow the existing code layout. That means 4-space indentation, ifs like if (condition) { ... and LF (\n) line endings (not Windows-like \r\n)
The one issue that I see currently is getting the TCL client to use SSL. I've done a PHP client, and I'm starting on a Java client, but I don't know TCL.
I'd be happy to help add SSL support to redis-rb
Cool. Googling, it seems that for the TCL side, we just need to use http://www.sensus.org/tcl/tls.htm#tls::import (but that may be over-simplifying)
@bbroerman30 Thanks for your great work here. Can you link the PHP or Java client patch as a reference for how to do SSL for Redis in a client? I'm thinking of doing a PR for https://github.com/garyburd/redigo if I can find some time.
I have a list here that includes Java and PHP: https://redislabs.com/blog/secure-redis-ssl-added-to-redsmin-and-clients - would love to have go there too
I've already done an ssl version of phpredis, and have been talking to the main developer for phpredis about merging my changes. They also require a change to php to fix a bug in their openssl extension. The pull request for php and phpredis has all been submitted... in any regards, i have the updates all available on my github account.
The PHP fix can be found at https://github.com/bbroerman30/php I have patches for 5.5 and 5.6 PHP as well as trunk. phpredis can be found at https://github.com/bbroerman30/ssl-phpredis
able to set up and run a non-ssl cluster now. Migrated to a different box with double the ram and a better CPU...
Is there a way of safely shutting down a cluster? I tried initially by shutting down each redis server instance, but bringing it back up didn't work well...
Just tried doing TLS via the tcl client. It's basically as simple as:
package require tls
proc redis_ssl {{server 127.0.0.1} {port 6379} {defer 0}} {
set fd [::tls::socket -require 0 -tls1 1 $server $port]
fconfigure $fd -translation binary
set id [incr ::redis::id]
set ::redis::fd($id) $fd
set ::redis::addr($id) [list $server $port]
set ::redis::blocking($id) 1
set ::redis::deferred($id) $defer
set ::redis::reconnect($id) 0
::redis::redis_reset_state $id
interp alias {} ::redis::redisHandle$id {} ::redis::__dispatch__ $id
}Though, this is only the client setup. I did not change any tests to launch a SSL Redis instance.
The above code is also highly insecure. It disables SSL checks (simply due to the fact that I test with self-signed keys ;))
Cool. It would be nice to set up some tests with that. I need a Ruby client to be able to run the cluster setup in SSL and test that functionality. I did basic tests on non-ssl cluster, though.
FYI, going forward I only have intermittent availability on weekends to work on this, until maybe summer, when I may be able to get a week off work.
I also need to update the redis-cli to take a domain name to test the certificate CN name against, and work on refactoring.
@bbroerman30 here's a PR to redis-rb that adds preliminary native SSL support:
Just tried running a redis cluster with SSL. They all started up well, but running ./redis-trib.rb it seemed to connect to the servers OK through SSL, but couldn't get them to join... Not sure if I'm missing something. I added the above PR to my copy of the gem (pasted the file), updated redis-trib to pass :ssl=>true on both instantiation requests and ran redis-trib.rb. I didn't see any errors in the logs. The first server responded with a lot of RET: 0 lines (meaning connects), but the others didn't seem to respond correctly... The output of the program is:
./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
Creating cluster
Connecting to node 127.0.0.1:7000: OK
Connecting to node 127.0.0.1:7001: OK
Connecting to node 127.0.0.1:7002: OK
Connecting to node 127.0.0.1:7003: OK
Connecting to node 127.0.0.1:7004: OK
Connecting to node 127.0.0.1:7005: OK
Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M: 00175d50ed1846e99b7f7a27197445b95904957c 127.0.0.1:7000
slots:0-5460 (5461 slots) master
M: 78c52917d1e5c9be14ddc9bd80c50cd720057ad3 127.0.0.1:7001
slots:5461-10922 (5462 slots) master
M: 79d9ee8a19f4f5674460521b8f1923aabb66ebc1 127.0.0.1:7002
slots:10923-16383 (5461 slots) master
S: b57d8d33b405a214d2d0ef8d89afb7b1ab2e4cf9 127.0.0.1:7003
replicates 00175d50ed1846e99b7f7a27197445b95904957c
S: fb9320754a23db44d9000449efe793d7bc5c6c89 127.0.0.1:7004
replicates 78c52917d1e5c9be14ddc9bd80c50cd720057ad3
S: 6d5d162083b9979df68441b56dcc6e865c8e4b29 127.0.0.1:7005
replicates 79d9ee8a19f4f5674460521b8f1923aabb66ebc1
Can I set the above configuration? (type 'yes' to accept): yes
Nodes configuration updated
Assign a different config epoch to each node
Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join......................................................................................................................
This just seemed to keep going until i killed it....
For each server, (other than 7000) I only see (7002 below, but the others are all similar):
332:M 16 Jan 14:39:10.523 * The server is now ready to accept connections on port 7002
3332:M 16 Jan 14:40:07.952 # RET: 0
3332:M 16 Jan 14:40:19.529 # configEpoch set to 3 via CLUSTER SET-CONFIG-EPOCH
@tarcieri Does the SLL connection for Ruby check the CN against the passed in value? If so, how does it handle when you pass in just an IP and port? Do you know if it will print errors for this, or for other SSL negotiation issues?
Looking over the cluster code, I am accepting and setting up SSL everywhere after a regular TCP accept (if ssl is enabled), similarly, I'm starting an SSL for write sockets... Not sure what's missing here.
@bbroerman30 it should verify the certificate by default and fail unless it's valid. You can pass in :ssl_params => {:ca_cert => 'ca.pem'} to provide your own CA
ok. I don't think it's the cert. It is connecting to the different servers and sending some information at least. It seems that it's not getting the MEET command sent, though. I don't even see an additional connect or messages on the different servers... just a lot of connect traffic on the 7000 server
I got it a little further along yesterday. It's trying to connect the cluster pieces together, but they're getting stuck while exchanging the MEET / PING to each other.
Got cluster working in SSL...
I have some clean up to do (get rid of a bunch of debug log statements I added) and some refactoring to do before it's ready for a pull request... Everyone feel free to test if you want (feedback is a gift)
Thanks for the ongoing work on that. I'll carefully review in the next weeks. As a rule of thumb, the more modular / isolated SSL is from the rest, the better of course: it makes for a much simpler merge, even if not 100% perfect, since it's opt-in and can be improved incrementally later. If instead it touches too many things there is the need to simplify and make it very robust before merging.
Unfortunately, it's not very modular or isolated. Essentially,
everywhere the file handle for the regular read/write was, I had to add
a struct that contained the SSL objects. Everywhere there was a read or
write call to a socket, I had to add an check for SSL and call the
OpenSSL libraries SSL_Read and SSL_Write methods instead of read /
write. I am planning on refactoring all of that out into a set of anet
methods to do the read and write for both SSL and regular connections,
and I'll also be doing some additional clean-up.
Since so much of the code refers to the fd itself (as an index or key) I
think the server / client structures should keep that and the sslCltn as
separate entities, though.
Bradley Broerman bbroerman@bbroerman.net
On Mon, Feb 2, 2015, at 04:37 AM, Salvatore Sanfilippo wrote:
Thanks for the ongoing work on that. I'll carefully review in the next
weeks. As a rule of thumb, the more modular / isolated SSL is from the
rest, the better of course: it makes for a much simpler merge, even if
not 100% perfect, since it's opt-in and can be improved incrementally
later. If instead it touches too many things there is the need to
simplify and make it very robust before merging.— Reply to this email directly or view it on GitHub[1].
Links:
@bbroerman30 are you able to open a pull request for this? I would like to circulate your code, and having a pull request where people can leave comments would be quite helpful
Sure. I haven't gone through any refactoring yet, but there's only a couple places that would mildly benefit from it.
If you and @antirez are good with what I have, I'll create one.
I'm sure there's a lot of work to be done yet, but having a PR would help in figuring out what the next steps are.
Done. PR #2402
I just found this article against putting SSL into severs. I've yet to digest / evaluate it, but looks like a counter argument worth linking here for further consideration: https://www.varnish-cache.org/docs/trunk/phk/ssl.html
There are some good arguments there, like he proposes keeping keys in a separate process, which is definitely best practice, but rarely implemented. Without it core dumps can contain SSL private keys, making them sensitive. And I don't think anyone disagrees that OpenSSL is a big, bloated, shaky piece of crap, although efforts like LibreSSL and BoringSSL seem promising.
This one is a bit overblown though:
We would terminate SSL sessions, and we would burn CPU cycles doing that. You can kiss the highly optimized delivery path in Varnish goodbye for SSL, we cannot simply tell the kernel to put the bytes on the socket, rather, we have to corkscrew the data through the SSL library and then write it to the socket.
There are reasons that encryption could be slow. Namely, it's slow if you do all three of these things:
- A client is disconnecting and reconnecting to Redis frequently
- That client isn't using session resumption
- That client is using TLS with RSA encryption, and the RSA operation is slow
However, if the client didn't do any of those things, or if the server switched from RSA to ECC, the performance problems are dramatically reduced.
Namely, once the handshake is complete, "corkscrew the data through the SSL library" on a modern x86-64 CPU should involve little more than calling AES-NI instructions on the data prior to telling the kernel to put the ciphertext on the socket. It'd be different if we tried to force a multiprocess model like PHK is suggesting, but as SSL is typically used/deployed (which is, indeed, less secure than what he's suggesting) it's pretty much a non-issue.
Hello Redis Community. First, please excuse my english/grammar if inaccurate. I have a few questions I was hoping those involved in this thread might be able to answer.
-
Will Redis SSL support make it into a stable 3.0.1+ release soon?
-
Once it does, will the following documentation references be updated for accuracy?
- http://redis.io/topics/security (Under Data encryption support)
- http://redis.io/topics/encryption (Possibly outdated/inaccurate especially now)
Using SPiped is just another tunnel mechanism which is discouraged or against the policies of some organizations, thus the requirement/need for full (native) SSL Support.
-
If the intention is for a secure communications channel from a Java Container (on Server A) to Redis (on Server B) is this SSL supported auth better or more appropriate for use (at least leveraging local host client to target host) than say the Redis Java Client aka 'JEDIS' (https://github.com/RedisLabs/jedis)? Has anyone explored both that they might be able to provide comparative analysis on it vs this pull?
-
Will the configurations allow to selectively choose SSLv3 vs TLS 1.0, 1.1, etc or will they simply negotiate automatically? (I have not yet tested it, but if someone has their input would be appreciated).
From what I understand, the main Redis admins want to refactor redis I/O and networking before making this a permanent feature. That's for antirez to comment
on. I assume at that time, all relevant documentation will be updated.
I have not worked with jedis as of yet, but it is in my queue when I have available time. I have, though used a custom version of phpredis that was updated to
use PHPs SSL streams.
Currently, my version uses the latest TLS available with the OpenSSL library. You can then specify in the configuration file which cyphers to use. With the way
the code in OpenSSL was available, this made the most sense at the time (especially since SSLv3 is vulnerable to so many attacks). If we want to allow
configuration options for older versions of the protocol, that certainly can be done.
Hope that helps,
Brad Broerman
On 4/17/2015 4:55 PM, Gatorade wrote:
Hello Redis Community. First, please excuse my english/grammar if inaccurate. I have a few questions I was hoping those involved in this thread might be able to
answer.
Will Redis SSL support make it into a stable 3.0.1+ release soon?
Once it does, will the following documentation references be updated for accuracy?
- http://redis.io/topics/security (Under Data encryption support)
- http://redis.io/topics/encryption (Possibly outdated/inaccurate especially now)
Using SPiped is just another tunnel mechanism which is discouraged or against the policies of some organizations, thus the requirement/need for full (native)
SSL Support.
If the intention is for a secure communications channel from a Java Container (on Server A) to Redis (on Server B) is this SSL supported auth better or more
appropriate for use (at least leveraging local host client to target host) than say the Redis Java Client aka 'JEDIS' (https://github.com/RedisLabs/jedis)? Has
anyone explored both that they might be able to provide comparative analysis on it vs this pull?Will the configurations allow to selectively choose SSLv3 vs TLS 1.0, 1.1, etc or will they simply negotiate automatically? (I have not yet tested it, but if
someone has their input would be appreciated).—
Reply to this email directly or view it on GitHub #2178 (comment).
@antirez - Would you be able to comment on ETA for when this might become a permanent feature in the stable release?
@Gatorade currently I'm not even sure if the feature will be accepted unfortunately. This is considered a low priority task, so basically I'll need to:
- Check the quality of the pull request.
- Check the potential quality of the PR, that is, there is a way to make this a separated, opt-in, layer at all? Or it will end always making everything more complex for the minority of users needing this?
After evaluating 1 & 2, there feature will either be accepted or rejected. If it is accepted it is possible that it will also require changes. So I've no idea about the ETA. It would be already a success if I could be able to get the time to inspect it, but I've currently more high priority stuff in queue waiting to be addressed.
@antirez any chance of getting this PR reviewed? I understand it's probably unmergable in its current state, but it would be good to have a list of action items / roadmap to getting it merged, even if that involves some prerequisite changes to e.g. Redis connection management.
I'd just be happy to see some comments that could be used as feedback to move this forward!
I thought I would add a comment since I have used Redis typically running on localhost for local applications that had a need for Redis, but have recently come across a need to set up an external Redis cluster (external to the application, not our network) for High Availability purposes. This would require encrypted communications from Application <=> Redis via the adapter/driver as well as Redis <=> Redis replication.
I agree with @tarcieri about the use of tunnel proxies not really being a solution here.
We have our own internal Certificate Authority to sign all of our certificates and I can at least speak about our recent setup/configuration of a MongoDB cluster which made it very easy to install SSL Key/Certs on every MongoDB server, along with our internal Root CA certificate for verification and to enforce that only authorized servers with valid certificates are authorized to connect. By enforcing certificates that are validated by the CA, this allows the server to validate certificates presented by connecting clients. It allows a member of the cluster to validate other cluster/replication members as it connects to them by hostname and validates the certificate. Then on top of this, one would still need to present either user/password (or via certificate) for authentication to specific data sets.
I think it would be great if Redis also had this capability as it would make things a lot more secure when you need to host or connect to a Redis server that is hosted externally from the application.
@antirez - Agreeing with both @tarcieri and @animedbz16, it seems clear there is distinct demand for the capabilities. Having Redis allow remote systems to authenticate over SSL/TLS is a growing demand by our partners who have stringent security requirements. This is especially important when the requesting client is in a separate data center from the Redis cluster itself and/or when it requires node-to-node encryption.
Related was PR #2402, but seems last reviewed/updated back in Feb/2015.
SSL/TLS support is a major blocker to using Redis for many users; I'm not sure that the mention at http://redis.io/topics/encryption of "the small percentage of users requiring SSL support" takes into account that the vast majority of teams looking at Redis who require SSL will not become users of Redis, since it doesn't meet their needs. This has been a major blocker to Redis adoption at three major companies I've worked with, one of which has a blanket policy preventing its use as anything but a localhost-only single-instance cache--no master/slave, no clustering, etc.
Given the state of network security today, the model of "have a secure firewall and trust everything on the network" is a pretty ineffective one, as the vast majority of successful attacks are piggybacking on VPN connections and valid openings in the firewall, i.e. by compromising an employee's laptop and connecting via their VPN connection when they are using it for legitimate purposes. Redis is seen in many organizations as an invitation for further compromise given the ease at which it allows critical data to be seen and modified without authorization beyond "got onto the production network."
The recent increase in discussion of this topic is good; if there are any ways to help prioritize this more highly among the wonderful folks with the time and skills to implement it, please let those of us who want to use Redis more know.
Throwing in my +1 here as well. I have a HIPAA project which requires all data in transit to be encrypted, even if it's within an internal network. I've been trying to get a Redis master/slave/sentinel deployment going with spiped, but to say it's a non-intuitive setup would be an understatement.
SSL support would be greatly appreciated by many of us ;-)
Just an FYI: I have updated my ssl-redis to include the latest stable releases of 2.8.24 and 3.0.6, and am working to merge in the latest 3.2 stable tag. In 2.8.24 and 3.0.6 I've tested basic communication, replication, and sentinel. I haven't been able to get the SSL version of the Ruby gem to work yet, so I've tested cluster in 3.0.6 without SSL, but not yet with it.. I've also updated the phpredis connector to be in sync with the latest release, and have an SSL enabled version of jedis available as well. Still working (in my spare time) to get these fully tested. I did run their internal test suite, though, in SSL as well as plain.
@bbroerman30 3.2 (which is very similar to unstable) port is a good news in order for the feature to be reviewed & eventually merged. During the latest two Redis meetings I received a lot of negative feedbacks about merging native SSL support into Redis, but yet I would merge it if the implementation is almost completely decoupled of the Redis core and just an opt-in at compile time, not enabled by default. I'll review your set of changes in the next weeks.
@bbroerman30 also, thanks for your efforts so far, that's great. This motivates me into looking into the result and evaluating with care if we can merge your work.
@antirez can you share any examples of the negative feedback you received during the Redis meetings?
I guess i am late to the party here, and not yet looked into the proposed implementation, looking forward to evaluating it, but throwing in my 5 cents to the discussion.
Why not implement this feature in the sentinel? My rationale being:
1 - separate process
2 - ssl usually only required for multiple hosts/clusters
3 - certs etc stored outside of redis - sentinel can handle all transport layer auth shenanigans.
4 - sentinel is a better place to automatically manage the localhost tunnels.
5 - little/no change in clients required other than sentinel compatibility which they need anyway.
6 - guessing here but assuming it is simpler to implement/maintain? (i see talk of having to re-factor the whole I/O layer for the current proposal)
@timoc Are you talking about Redis Sentinel?
@timoc then it would not help much at all. Clients still need to talk to individual Redis nodes and Sentinel itself talks to Redis nodes.
Understood, but in the scenario i envisage, each encrypted Redis node (and endpoint if not a local client) is running a Redis sentinel. Then the Redis sentinel manages the complexity of the transport layer security between encrypted hosts using key based authentication.
From the docs:
"Redis Sentinel is a monitoring solution for Redis instances that handles automatic failover of Redis masters and service discovery (who is the current master for a given group of instances?). Since Sentinel is both responsible to reconfigure instances during failovers, and to provide configurations to clients connecting to Redis masters or slaves, clients require to have explicit support for Redis Sentinel."
As i understand the sentinel mechanism, a redis client first connects to a sentinel to discover address of the master, then using the master information, it re-connects to the appropriate instance.
Again from the docs:
"A Redis client supporting Sentinel can automatically discover the address of a Redis master from the master name using Redis Sentinel. So instead of a hard coded IP address and port, a client supporting Sentinel should optionally be able to take as input:
A list of ip:port pairs pointing to known Sentinel instances. The name of the service, like "cache" or "timelines".This is the procedure a client should follow in order to obtain the master address starting from the list of Sentinels and the service name."
So in my scenario, the sentinel can create and maintain the encrypted tunnels to the various sentinel instances and clients, advertise the localhost tunnels mapping to the remote nodes.
the equivalent of managing something like:
ssh -R 6379:localhost:6379 senintel@remoteredisserverhost
This does not encrypt the data, or any overhead in the use of a local instance, only the case where i want to use Transport layer security on an encrypted cluster.
@timoc we don't use Sentinel nor want to: we already have shared infrastructure that accomplishes the same thing.
I think all parts of the Redis ecosystem should support SSL, rather than making you opt into Sentinel to be able to take advantage of it.
@bbroerman30 Thank you for all your work so far. For us, TLS/SSL is essential. One thing I wanted to bring up was OpenSSL engine support. Do you think it would be possible to add this, if it's not already present? Basically, we use hardware security modules that can expose themselves to OpenSSL through a custom engine. Would it be possible to specify the engine to use in the redis configuration file?
@swaj none of that is even relevant until #2402 is merged
@tarcieri That exact merge is where I'm asking for the code to be placed. If @bbroerman30 doesn't have the time or desire to add it himself, I'll happily contribute the necessary code.
I probably won't have time for a few months due to work... I can try to research it then. Can you point me to some relevant documentation?
Thanks!
Sent from my T-Mobile 4G LTE Device
-------- Original message --------From: Scott Anderson notifications@github.com Date: 4/22/16 5:08 PM (GMT-05:00) To: antirez/redis redis@noreply.github.com Cc: Brad Broerman bbroerman@bbroerman.net, Mention mention@noreply.github.com Subject: Re: [antirez/redis] Redis SSL support (#2178)
@tarcieri That exact merge is where I'm asking for the code to be placed. If @bbroerman30 doesn't have the time or desire to add it himself, I'll happily contribute the necessary code.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
@bbroerman30 You'll want ENGINE_set_default. There's good documentation for it here:
https://www.openssl.org/docs/manmaster/crypto/engine.html
Here's an example of how Node.js does the same thing:
https://github.com/nodejs/node/blob/master/src/node_crypto.cc#L5742
antirez, when can this SSL patch be merged it? Like many who initially look at Redis, we need security built into the software, not just an after thought. Please consider landing this work. You might be surprised by the uptick in community adoption once it's a built in feature.
Something tells me that SSL support in open-source Redis will lower the
value of hosting solutions, like Redis Labs and Azure cache.
Correct me if I'm wrong.
2016-08-08 20:10 GMT+03:00 MikesAracade notifications@github.com:
antirez, when can this SSL patch be merged it? Like many who initially
look at Redis, we need security built into the software, not just an after
thought. Please consider landing this work. You might be surprised by the
uptick in community adoption once it's a built in feature.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2178 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALnfJkv4LgFRzwv4BiWkTizNAj-DPUFks5qd2MggaJpZM4DCryh
.
Yours faithfully,
Gleb
So what?
It may mean that not going forward with SSL support is no longer merely a
techical matter, it's a business matter.
2016-08-09 18:59 GMT+03:00 MikesAracade notifications@github.com:
So what?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2178 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALnfP-06YLyq-b5t2bUNoPr8qiD2-ALks5qeKPsgaJpZM4DCryh
.
Yours faithfully,
Gleb
I have versions with SSL support in my own repos. They are a little out of date (I last updated them and brought them current in December) but you are free to use them. I will be updating them again with the latest stable release code around Christmas, when I have vacation again.
@chester89 its a security matter.
That's great news @bbroerman30. I will try and make Python client work with
SSL by then. Should be too hard
2016-08-09 19:27 GMT+03:00 Brad Broerman notifications@github.com:
I have versions with SSL support in my own repos. They are a little out of
date (I last updated them and brought them current in December) but you are
free to use them. I will be updating them again with the latest stable
release code around Christmas, when I have vacation again.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2178 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALnfCDFzsQi3_5d6RKj-0-BC4FFnRWuks5qeKpbgaJpZM4DCryh
.
Yours faithfully,
Gleb
@kvlou for users of Redis - yes, sure. I meant absense of SSL support in
open-source Redis version is a business decision for hosting and cloud
providers.
2016-08-09 19:29 GMT+03:00 Gleb Chermennov thebitterend77@gmail.com:
That's great news @bbroerman30. I will try and make Python client work
with SSL by then. Should be too hard2016-08-09 19:27 GMT+03:00 Brad Broerman notifications@github.com:
I have versions with SSL support in my own repos. They are a little out
of date (I last updated them and brought them current in December) but you
are free to use them. I will be updating them again with the latest stable
release code around Christmas, when I have vacation again.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2178 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALnfCDFzsQi3_5d6RKj-0-BC4FFnRWuks5qeKpbgaJpZM4DCryh
.Yours faithfully,
Gleb
Yours faithfully,
Gleb
Something tells me that SSL support in open-source Redis will lower the
value of hosting solutions, like Redis Labs and Azure cache.
I'd much rather we all have a standard way to do SSL rather than hacking up an stunnel solution which works differently across providers, or worse forking redis to add support thus drifting away from mainline, with all the risk associated with that. The way it is now, everyone's doing it differently so we're essentially fracturing the community as a result. If it were official, client libraries can follow suit and have a cohesive standard way to do it.
I agree. TLS support is long overdue.
@hgmnz there's already a way forward for clients: they should implement SSL/TLS natively, and accept rediss:// URLs:
https://www.iana.org/assignments/uri-schemes/prov/rediss
I recently added native SSL/TLS support to redis-rb. Several other Redis client libraries already support it as well.
@tarcieri ah! Thanks for that!
Dear Gleb,
Despite the nice conspirational ring that your theory has, it is entirely
wrong :)
On Aug 9, 2016 5:55 PM, "Gleb Chermennov" notifications@github.com wrote:
Something tells me that SSL support in open-source Redis will lower the
value of hosting solutions, like Redis Labs and Azure cache.
Correct me if I'm wrong.2016-08-08 20:10 GMT+03:00 MikesAracade notifications@github.com:
antirez, when can this SSL patch be merged it? Like many who initially
look at Redis, we need security built into the software, not just an
after
thought. Please consider landing this work. You might be surprised by the
uptick in community adoption once it's a built in feature.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2178 (comment),
or mute
the thread
<https://github.com/notifications/unsubscribe-auth/
AALnfJkv4LgFRzwv4BiWkTizNAj-DPUFks5qd2MggaJpZM4DCryh>
.Yours faithfully,
Gleb—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#2178 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFx1_N3NY5mVMpQUi2y-5lkA1ucA-Yd9ks5qeJT0gaJpZM4DCryh
.