mozilla / bigquery-etl

Bigquery ETL

Home Page:https://mozilla.github.io/bigquery-etl

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support rebuilding a rolling window of multiple table partitions via Airflow

sean-rose opened this issue · comments

Sometimes we need to rebuild more than just the latest partition (e.g. due to late arriving data).

┆Issue is synchronized with this Jira Task

Some examples of tables which could use this feature:

  • moz-fx-data-shared-prod.mozilla_vpn_derived.active_subscription_ids_v1
  • moz-fx-data-shared-prod.mozilla_vpn_derived.active_subscriptions_v1
  • moz-fx-data-shared-prod.mozilla_vpn_derived.subscription_events_v1
  • moz-fx-data-shared-prod.mozilla_vpn_derived.channel_group_proportions_v1
  • moz-fx-data-marketing-prod.ga_derived.www_site_metrics_summary_v2 (mozilla/telemetry-airflow#1891)