gjbae1212 / go-bqworker

go-esworker is an async worker that data can bulk insert, update to the BigQuery.

go-bqworker

go-esworker is an async worker that data can bulk insert, update to the BigQuery.
It's a library for golang.

Getting Started

Install

$ go get -u github.com/gjbae1212/go-bqworker

Usage

// pseudo code
package main
import (
    "context"
	"time"
    "github.com/gjbae1212/go-bqworker"
)

func main() {
	streamer, _ := bqworker.NewStreamer([]*bqworker.TableSchemaGroup{},
		bqworker.WithQueueSize(7000), bqworker.WithWorkerSize(6), bqworker.WithWorkerStack(500),
		bqworker.WithMaxRetry(2), bqworker.WithWorkerWaitDuration(time.Second),
		bqworker.WithDispatcherLoopWaitDuration(time.Hour), bqworker.WithErrorHandler(func(err error) {}))
	streamer.Start()
	
   // async
   streamer.AddRow(ctx, row)     
   // sync 
   streamer.AddRowSync(ctx, row)
}  

LICENSE

This project is following The MIT.