feature: `protoc-gen-go-grpc` to create test client / `bufconn` constructors

[Codey Oxley]

Summary

(Originally posted in golang/protobuf#1610)

My teams and I find ourselves writing test scaffolding like this a lot:

package main

import (
	"context"
	"net"
	"testing"

	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"
	"google.golang.org/grpc/test/bufconn"

	"github.com/stretchr/testify/require"

	pb "path/to/proto/package"
)

// NewClient returns a connected gRPC client to an in-memory service
func NewClient(t testing.TB, handler pb.Server) pb.Client {
	t.Helper()

	srv := grpc.NewServer()
	pb.RegisterServer(srv, handler)

	lis := bufconn.Listen(1 << 10)
	go func() {
		if err := srv.Serve(lis); err != nil {
			t.Logf("service exited with error: %v", err)
		}
	}()

	t.Cleanup(func() {
		srv.GracefulStop()
		lis.Close()
	})

	conn, err := grpc.NewClient(
		"bufnet",
		grpc.WithTransportCredentials(insecure.NewCredentials()),
		grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) {
			return lis.DialContext(ctx)
		}),
	)
	require.NoError(t, err)


	t.Cleanup(func() {
		conn.Close()
	})
	return pb.NewClient(conn)
}

We can create our own test helpers, but it's pretty unwieldy / ugly to use even with generics because of the generated factory functions: pb.RegisterServer, pb.NewClient. The signature needs to look something like this to get the appropriate type constraints:

func NewClient[S any, C any](
	t testing.TB,
	registerFn func(s grpc.ServiceRegistrar, srv S),
	newClientFn func(cc grpc.ClientConnInterface) C,
	handler S,
) C

I supposed we could make our protoc-gen-go-grpc-test, but given that google.golang.org/grpc/test/bufconn is a central package it doesn't seem completely out of place to ask for type-specific generation to make this easier.

An added benefit would be socializing in-memory testing of gRPC testing as an alternative to using mocks.

I'm not sure what the ideal signature would look like, but for conversation purposes:

// The generated file for proto package 'foo'
package foo

type FooTest struct{}
func (FooTest) Server() *grpc.Server
func (FooTest) Client() FooClient
func (FooTest) Close() error

func NewFooTest(handler FooServer, opts ...TestOption) (FooTest, error)

func WithTestDial(opts ...grpc.DialOption) TestOption
func WithTestServer(ops ...grpc.ServerOption) TestOption
func WithTestBuffer(size int) TestOption

Alternatively, this could be avoided if it was easier to use pb.RegisterServer and pb.NewClient in type constraints.

[Doug Fawley]

Thanks for the suggestion.

I think you can do this with a more general function instead. The function would accept a *grpc.Server that has been created by the test with the right pb.Regster* functions already called on it. It would create the bufconn listener and return a ClientConn connected to it. Then the test would do client := pb.FooClient(cc).

I.e.

package bufconntesting

func StartBufConnService(s *grpc.Server) (*grpc.ClientConn, error) {
	// (mostly just your code above:)
	t.Helper()

	lis := bufconn.Listen(1 << 10)
	go func() {
		if err := srv.Serve(lis); err != nil {
			t.Logf("service exited with error: %v", err)
		}
	}()

	t.Cleanup(func() {
		srv.GracefulStop()
		lis.Close()
	})

	return grpc.NewClient(
		"bufnet",
		grpc.WithTransportCredentials(insecure.NewCredentials()),
		grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) {
			return lis.DialContext(ctx)
		}),
	)
	require.NoError(t, err)
}

---

package some_test

func TestFoo(t *testing.T) {
	s := grpc.NewServer(<whatever opts this test case likes>)
	mypb.RegisterFooService(s)
	cc, err := StartBufConnService(s)
	if err != nil { t.Fatal() }
	client := pb.FooClient(cc)
	// use client
}

One problem you'll run into is that if a test case wants custom dial options, now you'll need to pass those in, too. (I avoided that problem by making the test create the server.) This is why I'm generally not a fan of testing helpers like this, although we do use them to some extent in our own tests. This and other opinions that you have in the code (e.g. to simply log the error if Serve exits with one, vs. calling t.Error) are reasons why we can't really be in the business of standardizing this kind of thing.

[Codey Oxley]

Yeah, fair enough!