Manos/HTTPS
opened this issue · comments
This is less of an issue and more of an improvement, actually: I think Manos could benefit from supporting HTTPS connections.
Yes definitely. It is on the roadmap and I've discussed it a little bit with Gonzalo. It shouldn't be too hard.
I'll post an updated roadmap on friday as part of my talk at confoo.
Any news here? Creating a SocketStream wrapper for System.Net.Security.SslStream should suffice to implement this, though I'm not quite sure how to expose TLS related things to the users.
I've just looked at the mono ssl code a bit, and it seems as though integrating that into Manos would be kind of hard. So apart from tunnelling TLS data through a decode process with another socket, I currently don't have the faintest on how to do this correctly and nice at the same time.
Thanks Myeisha,
Would you mind outlining what you think will be the hard parts? I
really haven't had a chance to look at it yet but would love to hear
your thoughts here.
On 3/27/11, myeisha
reply@reply.github.com
wrote:
I've just looked at the mono ssl code a bit, and it seems as though
integrating that into Manos would be kind of hard. So apart from tunnelling
TLS data through a decode process with another socket, I currently don't
have the faintest on how to do this correctly and nice at the same time.Reply to this email directly or view it on GitHub:
#60 (comment)
Sent from my mobile device
Not at all.
But I might have been a bit eager to announce difficulties. We could, theoretically, track a TLS connection instance with only the length field in each packet. Making sure that no packet is passed to the decryption layer while not yet complete should ensure that the decryption layer never blocks. What we would have to do, then, is buffer data until a packet is complete, decrypt that, and raise the Read event of the appropriate IOStream. It might just work, but that is a should: it's very sensible to assume that a TLS implementation will not attempt to read more than one packet from the source unless instructed to, but one can never be sure. I'd take the risk there and kill the session via some exception whenever the decryptor stream tries to read more than is available.
But even in that case, the intial handshake might be a problem. The handshake requires two (or more) rounds of message exchanges, and the secure streams in Mono block on handshake. So the handshake could be done in a thread, which isn't terribly nice, or it could be done blocking the rest of Manos, which is obviously not an option.
My guess is that the threaded handshake is the best way to go here. Adapting Mono's secure streams will be way too much work. Mono itself uses async IO for the handshake and only uses threads when data becomes available (and epoll to wait for that state on linux). It's not even that much of a pain in the arse to implement it in a way that hides every unpleasant detail from the user.
Yeah I think what we should do is sketch out the API and tooling from
a user point of view and then just see if we can bootstrap the whole
thing by using a threadpool and blocking I/O. In the future we can
always try to use the evented socket stream.
Would you be interested in designing the API?
On 3/27/11, myeisha
reply@reply.github.com
wrote:
But even in that case, the intial handshake might be a problem. The
handshake requires two (or more) rounds of message exchanges, and the secure
streams in Mono block on handshake. So the handshake could be done in a
thread, which isn't terribly nice, or it could be done blocking the rest of
Manos, which is obviously not an option.My guess is that the threaded handshake is the best way to go here. Adapting
Mono's secure streams will be way too much work. Mono itself uses async IO
for the handshake and only uses threads when data becomes available (and
epoll to wait for that state on linux). It's not even that much of a pain in
the arse to implement it in a way that hides every unpleasant detail from
the user.Reply to this email directly or view it on GitHub:
#60 (comment)
Sent from my mobile device
Sure thing :) I have a few ideas for that already, we just might be able to hide the SSL internals from the user completely.
I'm currently testing how scalable the builtin SSL streams actually are. We'd only have to revert to threadpool/blocking IO for the handshake phase anyway, so that shouldn't be too much of a problem in the end.
Excellent! Thanks. https will be a big step to 1.0
On 3/27/11, myeisha
reply@reply.github.com
wrote:
Sure thing :) I have a few ideas for that already, we just might be able to
hide the SSL internals from the user completely.I'm currently testing how scalable the builtin SSL streams actually are.
We'd only have to revert to threadpool/blocking IO for the handshake phase
anyway, so that shouldn't be too much of a problem in the end.Reply to this email directly or view it on GitHub:
#60 (comment)
Sent from my mobile device
Unfortunately, I seem to be unable to get SSL to work with Mono the way it works with every other server. Perhaps I'm just daft today, but is it really impossible to use PEM files for private keys in Mono?
Alright. I've continued my investigations and found a way to load at least something close to a sane format for certificates: PKCS12, with X509Certificate 2.
Unfortunately, at least on all machines I've tested my code on, Mono SSL is near to unusably slow. One handshake will take the better part of a second, or anything between that and a minute. It's insane. Sample: test code - takes up to three seconds for one handshake (and it's not CPU bound).
If that's not just me, we will have to wrap something around our sockets on the native level. It looks a lot like a bug in Mono, too ...
Thanks, I'll bring this up with Gonzalo (xsp, aspx author) tomorrow
and see what he says.
On 3/28/11, myeisha
reply@reply.github.com
wrote:
Alright. I've continued my investigations and found a way to load at least
something close to a sane format for certificates: PKCS12, with
X509Certificate 2.Unfortunately, at least on all machines I've tested my code on, Mono SSL is
near to unusably slow. One handshake will take the better part of a second,
or anything between that and a minute. It's insane. Sample: test
code
- takes up to three seconds for one handshake (and it's not CPU bound).
If that's not just me, we will have to wrap something around our sockets
on the native level. It looks a lot like a bug in Mono, too ...Reply to this email directly or view it on GitHub:
#60 (comment)
Sent from my mobile device
Ok, I've looked into this a little further. We could use GnuTLS quite easily, which I would prefer for speed reasons anyway. It's LGPL, so we'd have no license issues, it can parse standard certificate formats, and all that other nice stuff. All the while being pretty fast. Only problem is: it breaks sendfile, so we'd have to build a surrogate sendfile for TLS sockets. That could be done by sendfile-ing a file to a pipe pair, which is waitable for, or by using aio, which spawns threads. Ugly as it is, sendfile to a pipe is probably the best solution for this particular problem. What do you think?
I just had a brief chat with Sebastien and he agreed with your performance assesment. It shouldn't be quite as bad as you mentioned, but overall it sounds like we'd be better of with GnuTLS.
We should probably look at what node has done, I know they are using openssl and I believe they switched to it, but I have no idea why.
Unfortunately, OpenSSL has some licensing issues. In the end, it's probably a matter of personal conviction, but I prefer GnuTLS, if for the licensing alone. It also has some very nice support for asynchronous I/O, in that handshakes can be performed flawlessly over non-blocking sockets. I'm not sure how OpenSSL does there, but the GnuTLS API looks very nice and clean. I'll look at OpenSSL later, but GnuTLS is, as said, my favorite here.
It's coming to life. I now have an (untested) native backen for the TLS stuff, later I'll wrap a secure socket stream around that and unify SocketStream as it is now and the new, secure socket stream under some other abstract class. Shouldn't be long, if everything works out. :)
Awesome!
Sorry I've been a bit MIA the last couple of days and will be until
tomorrow evening. But I'm really excited about where you are going.
BTW, someone sent me this today:
http://www.mentalis.org/soft/projects/seclib/ might be useful not
sure. Haven't even had time to click the link :)
On Tue, Mar 29, 2011 at 11:21 PM, myeisha
reply@reply.github.com
wrote:
It's coming to life. I now have an (untested) native backen for the TLS stuff, later I'll wrap a secure socket stream around that and unify SocketStream as it is now and the new, secure socket stream under some other abstract class. Shouldn't be long, if everything works out. :)
Reply to this email directly or view it on GitHub:
#60 (comment)
Well, not so useful. I've found that library, too, but it's awfully old (roundabout ten years). The intended replacement has no support for SSL anymore, so it's a tad bit useless as well. Mono.Security does, what that library does. Probably fast, too, but it's still too slow (at least for me).