calogica / dbt-expectations

Port(ish) of Great Expectations to dbt test macros

Home Page:https://calogica.github.io/dbt-expectations/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature Request] Fast check that table is not empty

bochkarevnv opened this issue · comments

Is your feature request related to a problem? Please describe.
I need to check that table have any rows, but I don't see good expectation for that.

Describe the solution you'd like
I want fast probe if table has one row. If you want to check table not empty - you don't need to count all rows in table.

Describe alternatives you've considered
You can use expect_table_row_count_to_be_between with min_value 0, but it counts all rows in table, it can be very slow.

Additional context
In my case I write test with limiting number of rows selected for count. I think you can implement something like this.

{%- test expect_table_non_empty(model, row_condition=None) -%}
{{ default__test_expect_table_non_empty(model, row_condition) }}
{% endtest %}

{%- macro default__test_expect_table_non_empty(model, row_condition) -%}
with
    limited as (
        select 1
        from {{ model }}
        {%- if row_condition %} where {{ row_condition }} {% endif %}
        limit 1
    ),
    grouped_expression as (select count(*) as expression from limited),
    validation_errors as (select * from grouped_expression where not (expression > 0))

select *
from validation_errors

{%- endmacro -%}