datahub-project / datahub

The Metadata Platform for your Data Stack

Home Page:https://datahubproject.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

BigQuery issue: Tables with unapplied upsert data cannot be queried through prefix.

lethanhduong opened this issue · comments

Describe the bug
We're using Datastream to ingest the data to BigQuery. After that, we collect the BigQuery metadata to Datahub.
However, The ingestion job got a issue when it try to query partition information of Datasets using DataStream flow.

Error Log
400 Tables with unapplied upsert data cannot be queried through prefix. First table with upsert data

Script Called
The template query is executed in this case
https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/ingestion/source/bigquery_v2/queries.py#L72