A batch framework for golang.
There is Job as the top-level concept and WolfX instance bundles multiple jobs.
Each Job has a unique job name and target job can be executed by passing the job name to WolfX instance.
One or more Steps can be registered for a Job and executed in concurrently with each other.
Each Step has one reader and one writer.
The Reader and Writer runs on separate goroutines and they are connected by a channel.
Install the library first.
go get github.com/yackrru/wolfx
Note that the following sample code is incomplete.
DBToFileJob just works as a Job and it has one step, ReadAndOutputStep, which is set to execute sequentially.
The ReadAndOutputStep is incompletely configured,
but it can be read that the Reader is set to read data from the database and the Writer is set to write data to a file.
Finally, you can execute this job by passing DBToFileJob as an argument to Execute.
package main
import (
"context"
"github.com/yackrru/wolfx"
"github.com/yackrru/wolfx/integration/database"
"github.com/yackrru/wolfx/integration/file"
"github.com/yackrru/wolfx/middleware"
)
func Execute(jobName string) int {
wx := wolfx.New()
wx.Add(new(DBToFileJob))
if err := wx.Run(jobName); err != nil {
middleware.Logger.Fatal(err)
}
return 0
}
type DBToFileJob struct {}
func (j *DBToFileJob) Name() string {
return "DBToFileJob"
}
func (j *DBToFileJob) Run() error {
return wolfx.NewJobBuilder().
Single(j.ReadAndOutputStep).
Build()
}
func (j *DBToFileJob) ReadAndOutputStep(ctx context.Context) error {
return wolfx.NewStepBuilder(ctx).
SetReader(database.NewReader(&database.ReaderConfig{})).
SetWriter(file.NewWriter(&file.WriterConfig{})).
Build()
}The following can be used as Reader or Writer in Step.
| Type | Package.Name | Description |
|---|---|---|
| Reader | file.Reader | Reads a file using the file.CSVReader interface, which is satisfied by the standard package csv/Reader. |
| Writer | file.Writer | Writes a file using the file.CSVWriter interface, which is satisfied by the standard package csv/Writer. |
| Reader | database.Reader | Use sql/DB to load data with cursor from database. |
| Writer | database.Writer | Use sql/DB to import data to database. |