Yangruipis / go-functional

Golang lib for chain functional programming with spark-like APIs.

Go-Functional

Features

Go-functional is inspired by PyFunctional and go-funk. It makes golang struct data processing easier, especially when we have a data pipeline. Go-functionl is:

⚡️ FAST

there is no reflect at all

⛓️ CHAIN

chain functional operations

🎬 LAZY

most trainsformation operations are lazy evaluated

✨ SIMPLE

use generic, all you need is a Map , instead of MapInt MapInt32 MapInt64 …

👥 USER-FRIENDLY

Spark style APIs is provided, maybe LinQ someday.

Installation ( go>=1.18 )

go get github.com/Yangruipis/go-functional

Usage

basic import

 package main

 import (
   "fmt"

   "github.com/Yangruipis/go-functional/pkg/fn"
 )

 func main() {
   fn.RangeSeq(0, 10, 1).ForEach(func(i, v int) {
     fmt.Printf("%d\n", v)
   })
	}

Range -> Map -> Filter -> ForEach

fn.RangeSeq(0, 10, 1).Map(
  func(k, v int) (int, int) {return k, v + 1 },
).Filter(
  func(k, v int) bool { return v >= 3 },
).ForEach(
  func(i, v int) { fmt.Printf("%d\n", v) },
)

3
4
5
6
7
8
9
10

Range -> Map -> GroupByKey -> Aggregate -> ForEach

fn.RangeSeq(0, 10, 1).Map(
  func(k, v int) (int, int) {return k%2, v }, // split to 2 groups
).GroupByKey().Aggregate(func(vv []int) int {
  s := 0
  for _, v := range vv {
    s += v
  }
  return s
}).ForEach(
  func(k int, v int) { fmt.Printf("sum of group %d is: %v\n", k, v) },
)

mean of group 0 is: 20
mean of group 1 is: 25

multi path with cache: Range -> Map -> Cache ( –> Filter ) & ( –> Shuffle )

c := fn.RangeSeq(0, 10, 1).Map(
  func(k, v int) (int, int) { return k, v * 2 },
).Cache()

c1 := c.Filter(func(k, v int) bool { return v >= 10 } )
fmt.Printf("paths: %v, results: %v \n", c1.Paths, c1.ToSlice())

c2 := c.Shuffle()
fmt.Printf("paths: %v, results: %v \n", c2.Paths, c2.ToSlice())

paths: [Map Cache Filter], results: [10 12 14 16 18] 
paths: [Map Cache Shuffle], results: [2 14 8 0 18 4 6 10 16 12]

API list

There are two types of API(consistent with Spark):

1. Transformation: Iterator in, iterator out. Will not be executed until action operation. Iterator is supposed to be consumed only once.
2. Action: Iterator in, results out. All transformation operations are executed here (lazy exec).

func call[51/51]	chain call[33/51]	name	signature	type
✅	✅	Map	[K, V] -> [K, V]	transformation
✅	✅	Filter	[K, V] -> [K, V]	transformation
✅	✅	Flatten	[K, []V] -> [K, V]	transformation
✅	✅	GroupBy	[K, V] -> [K, []V]	transformation
✅	✅	GroupByKey	[K, V] -> [K, []V]	transformation
✅	❌	GroupByVal	[K, V] -> [V, []K]	transformation
✅	✅	FlatMap	[K, []V] -> [K, []V]	transformation
✅	✅	ReduceByKey	[K, []V] -> [K, V]	transformation
✅	✅	CountByKey	[K, V] -> [K, int]	transformation
✅	❌	CountByVal	[K, V] -> [V, int]	transformation
✅	❌	Union	[K, V] [K, V] -> [K, V]	transformation
✅	❌	Intersect	[K, V] [K, V] -> [K, V]	transformation
✅	❌	Subtract	[K, V] [K, V] -> [K, V]	transformation
✅	❌	Distinct	[K, V] -> [K, V]	transformation
✅	❌	UnionBy	[K, V] [K, V] -> [K, V]	transformation
✅	❌	IntersectBy	[K, V] [K, V] -> [K, V]	transformation
✅	❌	SubtractBy	[K, V] [K, V] -> [K, V]	transformation
✅	❌	DistinctBy	[K, V] -> [K, V]	transformation
✅	❌	Cartesian	[K, V] [K, V] -> [K, V]	transformation
✅	✅	Chunk	[K, V] -> [K, []V]	transformation
✅	✅	Sort	[K, V] -> [K, V]	transformation
✅	✅	Aggregate	[K, V] -> [K, V1]	transformation
✅	❌	Zip	[K, V] [K, V1]-> [K, [V, V1]]	transformation
✅	❌	Invert	[K, V] -> [V, K]	transformation
✅	✅	Reverse	[K, V] -> [K, V]	transformation
✅	✅	Shuffle	[K, V] -> [K, V]	transformation
✅	✅	Sample	[K, V] -> [K, V]	transformation
✅	✅	Choices	[K, V] -> [K, V]	transformation
✅	✅	Head	[K, V] -> [K, V]	transformation
✅	✅	Tail	[K, V] -> [K, V]	transformation
✅	✅	Cache	[K, V] -> [K, V]	transformation
✅	✅	Repeat	V -> [int, V]	transformation(init)
✅	✅	Range	int -> [int, int]	transformation(init)
✅	✅	Reduce	[K, V] -> V	action
✅	✅	Size	[K, V] -> int	action
✅	✅	Exists	[K, V] -> bool	action
✅	✅	ExistsBy	[K, V] -> bool	action
✅	✅	All	[K, V] -> bool	action
✅	✅	Any	[K, V] -> bool	action
✅	✅	Count	[K, V] -> int	action
✅	✅	CountBy	[K, V] -> int	action
✅	✅	ToSlice	[K, V] -> []V	action
✅	✅	ToMap	[K, V] -> map[K]V	action
✅	❌	ToSet	[K, V] -> map[K]struct{}	action
✅	❌	Sum	[K, V] -> int	action
✅	❌	Avg	[K, V] -> int	action
✅	✅	ForEach	[K, V] -> void	action
✅	✅	Entries	[K, V] -> [][K, V]	action
✅	❌	IndexOf	[K, V] -> []int	action
✅	❌	NIndexOf	[K, V] -> int	action
✅	✅	Values	[K, V] -> []V	action
✅	✅	Keys	[K, V] -> []K	action

Development

run tests

go mod download

go test ./...

contributing

• code format: gofmt
• commit format: see Angular Commit Message Conventions