googleapis / nodejs-bigquery

Node.js client for Google Cloud BigQuery: A fast, economical and fully-managed enterprise data warehouse for large-scale data analytics.

Home Page:https://cloud.google.com/bigquery/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Does createDataset support the supplying of a projectId?

atrbgithub opened this issue · comments

I'm trying to create a single instance of this client and then use it to create datasets in multiple projects.

For example:

const bigqueryClient = new BigQuery()

let ds = new Dataset(bigqueryClient, "a_test_dataset", {
   projectId: "a-test-project"
 })

ds.create()

This does not work, the dataset is created in whatever project is currently set as the default locally via gcloud.

const bigqueryClient = new BigQuery()
  bigqueryClient.createDataset("a_test_dataset", {
    projectId: "a-test-project"
  })

Same issue when doing it this way.

const bigqueryClient = new BigQuery()
  bigqueryClient.createDataset("a_test_dataset", {
    projectId: "a-test-project",
    datasetReference: {
      projectId: "a-test-project",
    },
  })

Finally this way throws an error:

Value some-default-project-id in content does not agree with value a-test-project. This can happen when a value set through a parameter is inconsistent with a value set in the request.

This is the only way that does work, but, it involves changing the projectId directly on the BigQuery client instance, it changes it for all subsequent requests.

  const bigqueryClient = new BigQuery()

  let ds = new Dataset(bigqueryClient, "a_test_dataset", {
     projectId: "a-test-project"
   })

   ds.bigQuery.projectId = "a-test-project"

   ds.create()

Interestingly, it does work for getDatasets:

  const bigqueryClient = new BigQuery()

  // @ts-ignore:next-line
  const [datasets] = await bigqueryClient.getDatasets({
    // @ts-ignore:next-line
    projectId: "a-test-project",
  })
  datasets.forEach((ds: any) => {
    console.log(ds.id)
  })

The above code returns the datasets from the correct project, although the types need to be overridden in typescript.

There was a PR from last year around cross project access. @steffnay @shollyman @feywind apologies for pinging you directly, do you know if creating a dataset should support the passing in of the projectId?