astronomer / astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

Home Page:https://astronomer.github.io/astronomer-cosmos/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature] Allow storing dbt ls cache into Object Store

tatiana opened this issue · comments

Description

Following up on #1014 , we'd like to allow users to store the dbt ls cache into a remote object store, as an alternative to store it in Airflow Variables. This task relates to #927.

Use case/motivation

Even though the dbt ls is being compressed without significantly affecting the Airflow Metadata database, some users prefer not to have cache information mixed up with their Airflow variables.

Related issues

Are you willing to submit a PR?

  • Yes, I am willing to submit a PR!