zees-dev / go-twirp

A lightweight rpc server that exposes HTTP 1.1 REST endpoints via use of Twirp

Go Twirp

A lightweight rpc server that exposes HTTP 1.1 REST endpoints via use of Twirp.

What is Twirp:

Twirp, an rpc based framework developed by twitch

Think of it as taking the best parts of gRPC and combining it with the simplicity of traditional REST APIs to give you best of both worlds - a high performance communication mechanism (RPC) that also supports REST clients.

For additional details, refer to the twitch blog post here

Why Twirp?

• When you want high performance communication between microservices (internal infrastructure) and also want to use the same models for external clients (web based)
  • Single proto files (models) can be used as source of truth for all internal and external services!
• When gRPC is too complex to setup/use for new developers and you need to hit the ground running - or those used to creating REST API's
• When you want to standardize your API endpoints - since theres a thousand ways to create REST API's (even though we have OpenAPI)
• REST APIs are not an efficient/optimal means of communication between your internal microservices

NOTE: gRPC-web can be used to achieve similar results; however it is non-trivial to setup (compared to Twirp) and requires a Envoy (gateway proxy) to translate web-based REST calls to gRPC.

Features

• RPC based web server that supports HTTP 1.1
• Hexagonal architecture
  • Loose coupling (everything is modular & testable)
• Support for both rest & protobuf based clients
• Docker support

Development

Pre-requisites

• protoc compiler - pre-compiled binary for your OS

• protoc-gen-go - protoc plugin for go

  • go get github.com/golang/protobuf/protoc-gen-go

• protoc-gen-twirp - protoc plugin for twirp

  • go get -u github.com/twitchtv/twirp/protoc-gen-twirp

• Note: All the dependencies must be available in your PATH

Generate code from protobuf compiler

protoc --proto_path=$GOPATH/src:. --twirp_out=. --go_out=. ./api/proto/todo/service.proto

Run Twirp based web-server

go run main.go

Clients

Clients can also be generated from proto files.
The magic of Twirp however is that it also supports HTTP 1.1 - hence allowing you to call RPC endpoints via curl (in addition to protobuf)

cURL

GET

curl -i \
  --request "POST" \
  --location "http://localhost:8080/twirp/ToDoService/Read" \
  --header "Content-Type:application/json" \
  --data '{"id": 1}'

POST

curl -i \
  --request "POST" \
  --location "http://localhost:8080/twirp/ToDoService/Create" \
  --header "Content-Type:application/json" \
  --data '{"toDo":{"title":"Second","description":"inital item in todo list"}}'

UPDATE

curl -i \
  --request "POST" \
  --location "http://localhost:8080/twirp/ToDoService/Update" \
  --header "Content-Type:application/json" \
  --data '{"toDo":{"id": "1", "title":"Second","description":"inital item in todo list"}}'

DELETE

curl -i \
  --request "POST" \
  --location "http://localhost:8080/twirp/ToDoService/Delete" \
  --header "Content-Type:application/json" \
  --data '{"id": 1}'

GET (get all)

curl -i \
  --request "POST" \
  --location "http://localhost:8080/twirp/ToDoService/ReadAll" \
  --header "Content-Type:application/json" \
  --data '{}'

Testing

go test ./... -v

Benchmarking

A benchmark has been provided which compares REST client performance against Protobuf client using provided benchmark file.
In this benchmark an in-memory HTTP server is initialized which serves the twirp API; we benchmark client REST and Protobuf requests against this API.

According to the benchmarks, the protobuf requests appear to be 10-15% faster than REST based client requests against the API.

go test -bench=.
goos: darwin
goarch: amd64
pkg: github.com/zees-dev/go-twirp
BenchmarkRestClient-8              13839             83063 ns/op
BenchmarkRPCClient-8               17268             69278 ns/op
PASS
ok      github.com/zees-dev/go-twirp    4.124s

Run benchmark

go test -bench=.

TODO

• Create/generate todo struct(s)
• Create DB (in-memory) to store db (this is mock)
• Complete REST handlers
• Implement interfaces and integrate with DB
• Readme completion
• Docker
• Testing
• Perform benchmarks to compare REST calls and native RPC calls on API
• Makefile
• Github actions (CICD)
• Goreport reference (badge)
• Complete documentation for public code (for godoc reference)
• Godoc reference (badge)
• Sourcegraph reference (badge)
• Build reference (badge)
• Code coverage reference (badge)

License