project status?
xiang90 opened this issue · comments
etcd team is working on etcd grpc layer proxy. For some rpc calls, we want to a go-through proxy just like what this project does. So I want to check the status of this project. Do you have plan to make this production ready? If yes, shall we collaborate on this one?
I don't know the status, but Mesosphere is also interested in a gRPC proxy that can intercept, parse, handle, and resend gRPC streams.
As far as I can tell, this project depends on a forked version of gRPC and the gRPC authors rejected the PR.
https://github.com/mwitkow/grpc-proxy/blob/master/proxy.go#L209
grpc/grpc-go#397
grpc/grpc-go#397 (comment)
So it looks like in order to work with gRPC a new ClientConn would need to be implemented to avoid using grpcConn.Picker().Pick(ctx).
@xiang90: have you made any progress on your own proxy?
@karlkfi We took a different approach by having a special grpc proxy just for etcd. Our most important use case requires a special proxy. We would love to revisit it if anyone wants to implement a generic proxy.
So you generated a server/client from the etcd proto and proxied at the gRPC message layer?
-Karl
On Jul 18, 2016, at 1:45 PM, Xiang Li notifications@github.com wrote:
@karlkfi We took a different approach by having a special grpc proxy just for etcd. Our most important use case requires a special proxy. We would love to revisit it if anyone wants to implement a generic proxy.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
@karlkfi Yes. https://github.com/coreos/etcd/blob/master/proxy/grpcproxy/maintenance.go#L61-L74. We can afford the overhead. But, again, I would love to collaborate on a more generic one if anyone can research the best way to implement it and do the initial work. I just do not really have enough bandwidth to start it effectively.
So I think it should be possible to proxy at the message layer generically, but it looks like it might require changes to gRPC-go. Currently gRPC-go does the unmarshalling itself, only giving the generated client methods an anonymous function wrapping the unmarshall call. If it provided some way to get the message byte array or Reader then you could proxy the message bytes without having to know what type to unmarshall/marshall to.
Proxying at the transport frame layer seems verboten based on the maintainer's response. It would require forking or rewriting the ClientConn which hides the transports and only exposes streams.
-Karl
On Jul 18, 2016, at 6:46 PM, Xiang Li notifications@github.com wrote:
@karlkfi Yes. https://github.com/coreos/etcd/blob/master/proxy/grpcproxy/maintenance.go#L61-L74. We can afford the overhead. But, again, I would love to collaborate on a more generic one if anyone can research the best way to implement it and do the initial work. I just do not really have enough bandwidth to start it effectively.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
Oh jeez, I completely missed this :) Sorry.
@karlkfi @xiang90 we are using this code internally for as frontend proxy/router. At Improbable we run it alongside a forked version of gRPC that has the Picker option (now GetTransport) added. It is true that the maintainer said they wouldn't want to expose the picker, but given the recent changes to the load balancing stack I think it's worth reaching out to @iamqizhao again.
I am going to think about this. My concern is that it should be for proxy usage only but we do not have a good way to stop abusing.
@iamqizhao We've been running this in production for half a year now, and we're find having gRPC proxies incredibly useful in a microservice architecture: we don't have to expose every service as a frontend, and rely on the proxy to do things such as auth and rate limiting.
I thought about moving the a layer up and implementing a generic ServerStream, and then using NewClientStream for proxying, but that would be less efficient and more hacky.
Maybe we could just name it GetLowLevelTransport and a comment: DoNotUseItMayBreak.
@mwitkow does the tap stuff that just landed in grpc-go provide an alternate implementation route?
You mean grpc/grpc-go#968? Sadly, no as it happens on the connection go-routine before any stream gets created.
There are really two changes to core gRPC that allow this to happen:
a) exposing the ClientConn.GetTransport (see here https://github.com/mwitkow/grpc-proxy/blob/master/patch/get_transport.go) to allow the Proxy to leverage all the nice load balancing features of gRPC without having to reimplement them, see discussion in (grpc/grpc-go#397)
b) Get an UnknownHandler for grpc-server so that you can register a "catch-all" proxy forwarder. See this PR grpc/grpc-go#685
Apart from that, we've been happily running it in our FE infrastructure at Improbable from our internal fork of grpc for a year now.
Right, the UnknownHandler has landed upstream, and the code has been rewritten to not require the low level picker access. This now works against upstream gRPC-Go.