Question about null fields

Question

Question about null fields

pedroteixeira opened this issue 10 months ago · comments

Pedro Henriques dos Santos Teixeira commented 10 months ago

Any particular reason why "Null fields are not supported"?
Any plans to support them?

Thanks,
Pedro

Timothée Peignier · Answer 1 · Fri Sep 29 2023 02:58:05 GMT+0800 (China Standard Time)

It's a limitation of Flink when using tuples, see: https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/dev/datastream/fault-tolerance/serialization/types_serialization/#flinks-typeinformation-class

David Reniz · Answer 2 · Fri Feb 09 2024 06:27:03 GMT+0800 (China Standard Time)

Hi, I came across this same issue... I understand this is a Flink limitation when using tuples, but:

How can we deal with nullable fields in Kotlin data classes? I think most non-trivial use cases would require some way to do so
Could TypeInfo for data classes be somehow modeled as a subclass of PojoTypeInfo instead of TupleTypeInfoBase? Flink documentation doesn't state POJOs have this same not-null limitation as Tuples do

PS: sorry to bring this up again, but seemed better to open a new issue

Timothée Peignier · Answer 3 · Fri Feb 09 2024 08:54:21 GMT+0800 (China Standard Time)

You can model nullable values as a distinct type that isn't nilable.

David Reniz · Answer 4 · Sat Feb 10 2024 06:49:24 GMT+0800 (China Standard Time)

Right, I thought that too. I tried using Java's Optional, but this can only be serialized via Flink Kryo fallback, which is not desirable. Do you see a better approach?