ibis-project / ibis-substrait

Ibis Substrait Compiler

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question about Ibis-substrait and DuckDB (NotImplementedError for AlchemyTable)

drin opened this issue · comments

I am playing around with ibis and duckdb and substrait and trying to piece together a concrete example and I am seeing:

NotImplementedError: <ibis.backends.base.sql.alchemy.database.AlchemyTable object at 0x12bda5e00>

The base table looks like:

r0 := AlchemyTable: expr
  gene_id !string
  cell_id !string
  expr    float64
>>> import ibis, ibis_substrait

>>> ibis.__version__
'4.0.0'

>>> ibis_substrait.__version__
'2.19.0'

I assumed that I could directly turn a query to DuckDB into substrait, but that does not seem to be the case. Should an unbound table always be used to generate substrait from ibis?

Here is a code sample:

import ibis

from ibis_substrait.compiler.core import SubstraitCompiler


db_conn = ibis.duckdb.connect(
     database='resources/exprdb.duckdb'
    ,read_only=False
)

table_obj = db_conn.table('expr')
print(table_obj)

substrait_compiler = SubstraitCompiler()
proto_msg = substrait_compiler.compile(table_obj)

Hi @drin!
So we don't have any translation rules for any ibis tables except for UnboundTable.

But (!), for ibis 4.x there's an unbind method on all expressions that will rewrite the expression with the same schema but with unbound tables.

unbind seems to work! thanks