Go RPC style implicit returns for replies.

[Dalton Hubble]

I'm trying to create an RPC service and expose it via go-thrift so that it supports a Go-RPC style method which takes args and a reply and returns an error. Critically, the modified reply struct should be available at the client. I don't think go-thrift currently allows this, but in a previous discussion, you hinted it could; maybe I'm doing something wrong.

type Cave struct{}

func (self *Cave) Echo(args *echoer_thrift.EchoArgs, reply *echoer_thrift.EchoReply) (error) {
    reply.Echo = args.Message
    return nil
}

And the Thrift definition,

struct EchoArgs {
    1: required string Message
}

struct EchoReply {
    1: required string Echo
}

service EchoerThriftface {
    void Echo(
        1: EchoArgs args
        2: EchoReply reply
    )
}

In a client,

var args = echoer_thrift.EchoArgs{}
args.Message = "hello!"
var reply echoer_thrift.EchoReply

conn, _ := net.Dial("tcp", ":1234")
thriftClient := thrift.NewClient(thrift.NewFramedReadWriteCloser(conn, 1024), thrift.NewBinaryProtocol(true, true), false)
client := &echoer_thrift.EchoerThriftfaceClient{thriftClient}
error := client.Echo(&args, &reply)

fmt.Println(reply.Echo)   // reply is EMPTY !!!

Looking at the code generated by go-thrift, it makes sense to my why this occurs. This is the expected behavior right?

[Samuel Stauffer]

Thrift doesn't explicitly define the request and response structs. They are implicitly generated based on the service definition in the IDL.

For example the service:

service EchoerThriftface {
    void Echo(
        1: EchoArgs args
        2: EchoReply reply
    )
}

When the call is made there are request and reponse structs that aren't defined but would look like it (if they were defined in the idl):

struct EchoRequest {
    1: required EchoArgs args
    2: required EchoReply reply
}

// The response is empty because the method returns void and doesn't define exceptions. But, for demonstration I show what it would look like with a return value and exceptions.

struct EchoResponse {
    // For responses other than void there would be a field id of 0
    // 0: optional SomeReply reply
    // For exceptions the id is 1+
    // 1: optional EchoFailException excFail
}

The code generator for Go in go-thrift generates these structs which can be used directly. For example the thrift idl you show would generate this: https://gist.github.com/samuel/7237130

Notice that EchoerThriftfaceEchoRequest and EchoerThriftfaceEchoResponse aren't defined in the IDL. They're the implicit structs/messages that thrift uses to represent a call and response.

Not sure if that helps to clear it up :P .. unfortunately the Thrift protocol is poorly documented.

[Dalton Hubble]

I see the implicit EchoerThriftfaceEchoRequest and EchoerThriftEchoResponse you mentioned. As you say, the EchoerThriftEchoResponse is empty which explains why nothing is returned.

If I were to use the service implementation

func (self *Cave) Echo(args *echoer_thrift.EchoArgs) (reply *echoer_thrift.EchoReply, error) {
    reply := &echoer_thrift.EchoReply{}
    reply.Echo = args.Message
    return reply, nil
}

then EchoerThriftEchoResponse would nicely contain the reply and the generated client would return that reply.

One of my constraints however is that I am unwilling to modify the implemented service method signature (or rather I don't want developers to have to modify services)

func (self *Cave) Echo(args *echoer_thrift.EchoArgs, reply *echoer_thrift.EchoReply) (error) {
    reply.Echo = args.Message
    return nil
}

I believe these implicit Request and Response structs are introduced to support general Thrift method calls which can have numerous arguments and pack them all up in a nice Request struct? In my case, since my Echo method conforms to the more constraining Go RPC format func (t *T) MethodName(argType T1, replyType *T2) error, it would be more useful if argType and replyType were used as the Request and Response types directly.

My generated code did match your gist previously. I've hacked the generated code a bit to make my desired service implementation work as I wanted. https://gist.github.com/dghubble/7238724 Thoughts?

And definitely agreed, thrift-world has been a bit scary sometimes. :)

[Samuel Stauffer]

Yeah, the default generated code is designed to work with existing Thrift implementations while giving an interface that looks Thrift-like.

You can send/receive arbitrary structs instead of using the implicit ones, but it would break compatibility with other thrift libraries. For instance, the response struct must have ID 0 as the return value of the method and ID 1+ as exceptions, and the request struct doesn't allow optional fields.

If you're in a pure Go-world than just using net/rpc directly and letting it do GOB encoding might make more sense.

However, if what you want is compatibility with Thrift in general and just to use the Go style of Call(req, res) then use the generated code but instead of using the Client methods just do the work directly.

The code in that thrift actually is what should be generated by the IDL:

type EchoArgs struct {
    Message string `thrift:"1,required" json:"Message"`
}

type EchoReply struct {
    Echo string `thrift:"1,required" json:"Echo"`
}

service EchoerThriftface {
    EchoReply Echo(
        1: EchoArgs args
        2: EchoReply reply
    )
}

So, it's actually serializing and sending the reply over the wire.. and the return value is where the value actually comes back. Meaning, the reply in the arguments to Echo isn't being used for anything.

This may be what you're looking for:

type EchoArgs struct {
    Message string `thrift:"1,required" json:"Message"`
}

type EchoReply struct {
    Echo string `thrift:"1,required" json:"Echo"`
}

service EchoerThriftface {
    EchoReply Echo(
        1: EchoArgs args
    )
}

Then you could modify the generated code to be:

func (s *EchoerThriftfaceClient) Echo(args *EchoArgs, reply *EchoReply) error {
    req := &EchoerThriftfaceEchoRequest{
        Args:  args,
    }
    res := &EchoerThriftfaceEchoResponse{
        Value: reply,
        }
    return s.Client.Call("Echo", req, res)
}

[Dalton Hubble]

Thanks, that's helpful!
I'm not working in a pure Go-world so I would like to maintain compatibility with other thrift libraries. I haven't seen this "For instance, the response struct must have ID 0 as the return value of the method and ID 1+ as exceptions, and the request struct doesn't allow optional fields." documented anywhere.

I'm mostly concerned with keeping my existing service signature of func (t *T) MethodName(argType T1, replyType *T2) error. I'm working on a library to make it super easy to define a service once, in the above restrictive manner, and then easily use a plugin thrift server, Go rpc server, protocol buffer, whatever server from the library.

I know the reply is duplicated here

service EchoerThriftface {
    void Echo(
        1: EchoArgs args
        2: EchoReply reply
    )
}

but it does generate in an interface with the desired signature:

type EchoerThriftface interface {
    Echo(args *EchoArgs, reply *EchoReply) error
}

whereas

service EchoerThriftface {
    EchoReply Echo(
        1: EchoArgs args
    )
}

generates

type EchoerThriftface interface {
    Echo(args *EchoArgs) (*EchoReply, error)
}

:( so I would have to wrap this in an interface that matches my desired interface and make more significant modifications.

I believe the 3 modifications I made in my previous gist maintain compatibility with thrift libraries in general.