using Overwrite flag on save injest the key only

[shadibch]

We tried to use the following code:
aggregations.filter("id is not null").groupBy("id").agg(collect_set("imsi")). write
.format("org.apache.spark.sql.redis")
.option("table", "tacs")
.mode(SaveMode.Overwrite)
.option("ttl", 300)
.option("key.column", "id")
.save()

This saves the key only. We should use the Append mode flag to save the dataframe

[Oleksii Diagiliev]

Hi @shadibch ,
could you please the sample dataframe (after the transformation) that you try to save?

[shadibch]

Hello
The problem is not related with the dataframe
Once we use the tag SaveMode.Overwrite) in any dataframe to be saved into redis it saves only the key.

[Oleksii Diagiliev]

Could you provide some example I can reproduce please?

[M Sazzadul Hoque]

Closed due to inactivity.