mgosk / spark-incremental-merge

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Spark Incremental Merge

3 ways of merging incremental updates in Spark.

  • Merge join whole dataset using dataframes
  • Merge join whole dataset using SparkSQL
  • Merge updated partitinos only using dataframes

About


Languages

Language:Scala 100.0%