ReadSession: selectedFields on null record coalesced to 0 value

[Michel Davit]

Environment details

OS type and version: Linux, Ubuntu 22.04
Java version: 11
version(s): 2.41.1, 2.44.0

Steps to reproduce

Considering this table

With the following data

Reading the table with AVRO format without any selected fields gives the expected result:

{"required": {"int": 0, "string": "0"}, "optional": null, "repeated": []}
{"required": {"int": 1, "string": "1"}, "optional": {"int": 1, "string": "1"}, "repeated": [{"int": 1, "string": "1"}]}
{"required": {"int": 2, "string": "2"}, "optional": {"int": 2, "string": "2"}, "repeated": [{"int": 2, "string": "2"}]}
{"required": {"int": 3, "string": "3"}, "optional": {"int": 3, "string": "3"}, "repeated": [{"int": 3, "string": "3"}]}
{"required": {"int": 4, "string": "4"}, "optional": {"int": 4, "string": "4"}, "repeated": [{"int": 4, "string": "4"}]}
{"required": {"int": 5, "string": "5"}, "optional": {"int": 5, "string": "5"}, "repeated": [{"int": 5, "string": "5"}]}
{"required": {"int": 6, "string": "6"}, "optional": {"int": 6, "string": "6"}, "repeated": [{"int": 6, "string": "6"}]}
{"required": {"int": 7, "string": "7"}, "optional": {"int": 7, "string": "7"}, "repeated": [{"int": 7, "string": "7"}]}
{"required": {"int": 8, "string": "8"}, "optional": {"int": 8, "string": "8"}, "repeated": [{"int": 8, "string": "8"}]}
{"required": {"int": 9, "string": "9"}, "optional": {"int": 9, "string": "9"}, "repeated": [{"int": 9, "string": "9"}]}

When now setting the selected fields to ["required", "optional.int"], the library returns the following

{"required": {"int": 0, "string": "0"}, "optional": {"int": 0}}
{"required": {"int": 1, "string": "1"}, "optional": {"int": 1}}
{"required": {"int": 2, "string": "2"}, "optional": {"int": 2}}
{"required": {"int": 3, "string": "3"}, "optional": {"int": 3}}
{"required": {"int": 4, "string": "4"}, "optional": {"int": 4}}
{"required": {"int": 5, "string": "5"}, "optional": {"int": 5}}
{"required": {"int": 6, "string": "6"}, "optional": {"int": 6}}
{"required": {"int": 7, "string": "7"}, "optional": {"int": 7}}
{"required": {"int": 8, "string": "8"}, "optional": {"int": 8}}
{"required": {"int": 9, "string": "9"}, "optional": {"int": 9}}

The 1st record is expected to return {"required": {"int": 0, "string": "0"}, "optional": null} instead. Somehow its value gets coalesced into 0.

[Michel Davit]

For reference, scala code gist to reproduce

[Michel Davit]

Any update on this ?

[emkornfield]

CC @kmjung

[Kenneth Jung]

I've been able to reproduce this issue, and have filed a tracking bug internally to continue the investigation.