TobikoData / sqlmesh

Efficient data transformation and modeling framework that is backwards compatible with dbt.

Home Page:https://sqlmesh.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Don't create temp tables at plan time unless they're actually needed

georgesittas opened this issue · comments

commented

We currently create temporary tables at plan time preemptively, even though they may not necessarily be needed. This behavior was introduced before we started to create tables on demand (i.e., if they're missing), but we can move away from it now.

For instance, the initial plan application for the example project produces the following tables in duckdb:

CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__full_model__3031530868(item_id INTEGER, num_orders BIGINT);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__full_model__3031530868__temp(item_id INTEGER, num_orders BIGINT);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__incremental_model__141311533(id INTEGER, item_id INTEGER, event_date DATE);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__incremental_model__141311533__temp(id INTEGER, item_id INTEGER, event_date DATE);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__seed_model__1248844913(id INTEGER, item_id INTEGER, event_date DATE);
CREATE TABLE sqlmesh__sqlmesh_example.sqlmesh_example__seed_model__1248844913__temp(id INTEGER, item_id INTEGER, event_date DATE);