Introduction

        .,*/(#####(/*,.                               .,*((###(/*.          
    .*(%%%%%%%%%%%%%%#/.                           .*#%%%%####%%%%#/.       
  ./#%%%%#(/,,...,,***.           .......          *#%%%#*.   ,(%%%#/.      
 .(#%%%#/.                    .*(#%%%%%%%##/,.     ,(%%%#*    ,(%%%#*.      
.*#%%%#/.    ..........     .*#%%%%#(/((#%%%%(,     ,/#%%%#(/#%%%#(,        
./#%%%(*    ,#%%%%%%%%(*   .*#%%%#*     .*#%%%#,      *(%%%%%%%#(,.         
./#%%%#*    ,(((##%%%%(*   ,/%%%%/.      .(%%%#/   .*#%%%#(*/(#%%%#/,       
 ,#%%%#(.        ,#%%%(*   ,/%%%%/.      .(%%%#/  ,/%%%#/.    .*#%%%(,      
  *#%%%%(*.      ,#%%%(*   .*#%%%#*     ./#%%%#,  ,(%%%#*      .(%%%#*      
   ,(#%%%%%##(((##%%%%(*    .*#%%%%#(((##%%%%(,   .*#%%%##(///(#%%%#/.      
     .*/###%%%%%%%###(/,      .,/##%%%%%##(/,.      .*(##%%%%%%##(*,        
          .........                ......                .......

A starter kit for Go API development. Inspired by How I write HTTP services after eight years. However I wanted to use chi router which is more common in the community, sqlboiler to solve database operations and design towards more like Clean Architecture.

This kit tries to follow the Standard Go Project Layout to make project structure familiar to a Go developer.

It is still in early stages and I do not consider it is completed until all integration tests are done.

In short, this kit is a Go + Postgres + Chi Router + SqlBoiler starter kit for API development.

Motivation

On the topic of API development, there are two opposing camps between a framework (like echo, gin, buffalo and starting small and only add features you need. However , starting small and adding features aren;t that straightforward. Also, you will want to structure your project in such a way that there are clear separation of functionalities for different files. This is the idea behind Clean Architecture. This way, it is easy to switch whichever library to another of your choice.

This project serves as a starting point. Further integrations such as elasticsearch, jobs, and authentication are in separate branches.

Features

This kit is composed of standard Go library together with well known libraries to manage things like router, database query and migration support. Technically it supports other databases as well.

Router/Mux with Chi Router
Database Operations with Sqlboiler ORM
Database migration with golang-migrate
Cache result with Redis stored with msgpack encoding
Input validation that return multiple error strings
Scans and auto-generate Swagger docs using a declarative comments format
Request log that logs each user uniquely based on host address
Cors
Pagination through middleware

It has few dependencies and replacing one library to another is easy.

Getting It

git clone https://github.com/gmhafiz/go8
cd go8

Quickstart

A. Have both a postgres database and a redis(optional) instance ready.

If not, you can run the following command if you have docker-compose installed:

docker-compose up -d postgres redis

B. This project uses Task to handle various tasks such as migration, generation of swagger docs, build and run the app. It is essentially a sh interpreter . Only requirement is to download the binary and append to your PATH variable.

Install task runner binary bash script:

scripts/install-task.sh
And put this binary in your path if not exists

echo 'PATH=$PATH:$HOME/.local/bin' >> ~/.bashrc source ~/.bashrc

Task tasks are defined inside Taskfile.yml file. A list of tasks available can be viewed with:

task -l   # or
task list

Once Task is installed, setup can be initiated by the following command:

task init

This copies example configurations for the app, sqlboiler and Task to its respective .yml files as well as syncs dependencies. Check your .env and sqlboiler.toml files.

C. Finally, to run,

task run

And you'll get the following log messages

task: go run cmd/go8/main.go
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:26 > starting at 0.0.0.0:3080 service=go8
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:86 >  routes={"method":"GET","path":"/api/v1/authors/"} service=go8
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:86 >  routes={"method":"GET","path":"/api/v1/books/"} service=go8
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:86 >  routes={"method":"POST","path":"/api/v1/books/"} service=go8
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:86 >  routes={"method":"GET","path":"/api/v1/books/{id}"} service=go8
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:86 >  routes={"method":"PUT","path":"/api/v1/books/{id}"} service=go8
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:86 >  routes={"method":"DELETE","path":"/api/v1/books/{id}"} service=go8
2020-10-09T11:44:50+11:00 INF internal/server/rest/server.go:86 >  routes={"method":"GET","path":"/swagger"} service=go8

To use, follow examples in the examples/ folder

curl --location --request GET 'http://localhost:3080/api/v1/books'

Migration

Migration is a good step towards having a versioned database and makes publishing to a production server a safe process.

Create Migration

Using Task, creating a migration file is done by the following command. Name the file after NAME=.

task migrate-create NAME=create_a_tablename

Migrate up

After you are satisfied with your .sql files, run the following command to migrate your database.

task migrate

Further golang-migrate commands are available in its documentation (postgres)

Database Generate Models and ORMs

SqlBoiler treats your database as source of truth. It connects to your database, read its schema and generate appropriate models and query builder helpers written in Go. Utilizing a type-safe query building allows compile-time error checks.

Generate ORM with:

task gen-orm

Generated files are as defined in the sqlboiler.toml file. This command needs to be run after every migration changes are done. Generated files are located in internal/model

Test

Install testify testing framework with

go get github.com/stretchr/testify

Run

Local

Conventionally, all apps are placed inside the cmd folder.

Using Task:

task run

Without Task

go run cmd/go8/go8.go

Docker

You can build a docker image with the app with its config files. Docker needs to be installed beforehand.

 task docker-build

Run the following command to build a container from this image. --net=host tells the container to use local's network so that it can access local's database.

task docker-run

Docker Compose

If you have docker-compose installed, you may run the app with the following command. Docker -compose binary must be installed beforehand.

docker-compose up -d

Both Postgres and redis ports are mapped to local machine. To allow api container to reach the database and redis.

Swagger docs

Swagger UI allows you to play with the API from a browser

Edit cmd/go8/go8.go main() function host and BasePath

// @host localhost:3080
// @BasePath /api/v1

Generate with

task swagger

Access at

http://localhost:3080/swagger

The command swag init scans the whole directory and looks for swagger's declarative comments format.

Custom theme is obtained from https://github.com/ostranme/swagger-ui-themes

Tooling

Various tooling are included within the Task runner

task fmt
- Runs go fmt ./... to lint Go code
- go fmt is part of official Go toolchain that formats your code into an opinionated format.
task tidy
- Runs go mod tidy to sync dependencies.
task vet
- Quickly catches compile error.
task golint
- Runs an opinionated code linter from https://golangci-lint.run/

Structure

This project mostly follows the structure documented at Standard Go Project Layout.

In addition, this project also tries to follow Clean Architecture where each functionality are separated into different files.

Starting Point

Starting point of project is at cmd/go8/main.go

Purpose of this is to initialize all dependencies (logger, read .env, validation, database, redis) and passes it on to the rest package. After all domains are initialized, the rest server can be started.

Domain

Looking at the internal/domain/books folder:

handler.go handles request coming in to perform input validation, calls usecase, and formats the output.
routes.go defines available routes for books.
usecase.go receives request from handler, perform business logic, and call repository if needed.
repository.go handles database operations. Also may queries redis cache.
cache.go performs redis queries.

Configurations

All environment variables are read into specific structs initialized in configs/configs.go.

Database

Migrations are stored in database/migrations folder

Libraries

Initialization of external libraries are located in internal/library

TODO

Complete HTTP integration test
use xID for table ID primary key
better control of json output formatting

go get github.com/golang/protobuf/protoc-gen-go go get google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.0

gmhafiz / masuk