A new, arguably faster, implementation of Apache Spark from scratch in Rust. WIP
Framework tested only on Linux, requires nightly Rust. Read how to get started in the documentation.
- Error Handling(Priority)
- Fault tolerance
Most of these except file reader and writer are trivial to implement
- map
- flat_map
- filter
- group_by
- reduce_by
- distinct
- count
- take_sample
- union
- glom
- cartesian
- pipe
- map_partitions
- for_each
- collect
- reduce
- fold
- aggregate
- take
- first
- sample
- zip
- save_as_text_file (can save only as text file in executors local file system)
- Replace hard coded values