Giters

delta-io / delta-sharing

An open protocol for secure data sharing

Home Page:https://delta.io/sharing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Protocol Spec - Predicate hints from DataBricks don't match the described description

rustyconover opened this issue · comments

commented

When querying with DataBricks for a partition column requests are made with

 '{"predicateHints":["((CAST(partitionValues.customer_id AS STRING) > \'A\') 
OR 
(NOT (CAST(partitionValues.customer_id AS STRING) = \'Z\')))"]}'

This predicate expression does not match the protocol spec. Is it assumed that there is a CAST() function?

commented

Spark connectors + standalone servers support a superset of the protocol, but we are deprecating predicateHints in favor of jsonPredicateHints. We will also ensure that both the connector and server will adhere to the protocol going forward.