Proper Null Handling

[Ethan]

Problem Statement

Currently, query resolution may sometimes drop rows depending on query path selected due to the requirement to merge CTEs on keys. If the keys have null values, this can result in unexpected loss of data.

This can be mitigated at the definition layer by adding a rawsql coalesce to any column with tainted values, but it would be ideal to support resolution on columns with nulls.

Proposed Solution

Add a concept level metadata flag for if it is nullable or not. (maybe copy type syntax?)

eg

key concept_id int | null;

When resolving queries with this, if the concept is used in a join, the join condition must be

[a.x = b.x OR a.x is null and b.x is null]

Alternatives Considered

Please briefly describe which alternatives, if any, have been considered, including merits of alternate approaches and
tradeoffs being made.

Additional Context

Please provide any other information that may be relevant.

[Ethan]

Implemented as ?, can update in future.

key concept_id int?