Log select/exclude arguments with invocations
falonso-alo opened this issue · comments
Describe the feature
It would be useful to log run select
/exclude
arguments. For example, if the following commands were run:
# run a single model
dbt run --select mymodel
# run a single tag
dbt run --select tag:mytag
# run a single tag with an exclude
dbt run --select tag:mytag --exclude no_not_that_model
Then it would be nice if the select
and exclude
arguments were logged as well. For example:
invocation_id | event_name | select | exclude |
---|---|---|---|
1 | run started | mymodel | null |
2 | run started | tag:mytag | null |
3 | run started | tag:mytag | no_not_that_model |
Of course, the columns should probably not be SQL keywords, but the exact column name wording is probably out of scope the request.
Describe alternatives you've considered
It might be possible to use a pre-hook to fetch that information an insert record in a table somewhere in the database. However, there are some downsides:
- This would be cumbersome
- It's unclear if this should go into the existing
dbt_meta.dbt_audit_log
table - This fits thematically with the logging package
Additional context
When there are potentially multiple DBT jobs running, it can be difficult to pin point which job is related to which invocation. Logging the select
/exclude
arguments would make it easier to tie the logging to a particular job.
Who will this benefit?
People who have multiple DBT runs every day that target different tags/models.