#GoMR - Goroutine based MapReduce
GoMR is toy-ish map reduce framework that can run MapReduce jobs locally across goroutines. It was written as an expriment in go, but put out here since a co-worker had asked to use it for some SmallData crunching, and I hope it might be useful to other people.
In src/example.go you can find the obligatory word counting example (that reads from stdin and counts words), but here is the gist of it:
To run a MapReduce job, you have to define a struct that implements GoMR's MapReducer interface that has:
1. A Mapper function of the form:
Map(input interface{}, out chan Record)(Record is simply a key,value pair, where the value can be anything). The mapper takes one piece of input, and maps to to N key/value pairs, pushing them to the out channel
2. A Reducer function of the form:
Reduce(key string, values []interface{}, out chan Record)The reducer takes a key, and a slice of values that are created by the mapper, and pushes back K {key,value} records to the output channel
3. An input reader function that returns one item to be processed (e.g. a document) per each call
Read() (record interface{}, err error)if it returns an error, the reading stops
###4. An output handler function of the form
HandleOutput(output chan Record)This function should consume the output of the job from the output channel completely