gcsfuse does not correctly use default credentials stored

[andylamp]

Describe the bug

gcsfuse is not able to discover default credentials stored in my machine. gcloud works as expected, I have provided sample outputs with sensitive information redacted.

➜  templates git:(main) ✗ gcsfuse --foreground /my-bucket ./my-mount
2023/02/07 19:06:58.837705 Start gcsfuse/0.41.12 (Go version go1.18.4) for app "" using mount point: /**redacted**/my-mount
2023/02/07 19:06:58.837770 Opening GCS connection...
2023/02/07 19:06:58.844308 Waiting for connection: GetTokenSource: DefaultTokenSource: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
2023/02/07 19:06:59.845170 Waiting for connection: GetTokenSource: DefaultTokenSource: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
2023/02/07 19:07:01.346371 Waiting for connection: GetTokenSource: DefaultTokenSource: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
^C
➜  templates git:(main) ✗ gcloud storage ls
gs://bucket-a/
gs://bucket-b/
.
.
.
gs://another-bucket/

System (please complete the following information):

• OS: Ubuntu 22.04.1 LTS
• Platform: VM
• Version:
  • gcsfuse --version
    gcsfuse version 0.41.12 (Go version go1.18.4)
  • gcloud --version
    Google Cloud SDK 416.0.0
    alpha 2023.01.30
    beta 2023.01.30
    bq 2.0.84
    bundled-python3-unix 3.9.16
    core 2023.01.30
    gcloud-crc32c 1.0.0
    gsutil 5.19

Additional context

I have also tried to re-authorise through gcloud auth login again, which was successful but the same error occurred.

[Prince Kumar]

Thanks andylamp@ for reaching out and providing the details!

Yes, you are right. GCSFuse is looking for application default credentials and not able to find.

Can you please try using gcloud auth application-default login command once?

Mostly the above command will place the application default credentials at $HOME/.config/gcloud/application_default_credentials.json. You can check.

Please let me know, if the above doesn't work.

[andylamp]

hello @raj-prince, thanks for the reply; it seems that using this worked. It'd be great if it was visibly in the readme so we know this a-priori.

happy for the issue to be closed as it is solved.

[Samuel Polo Peña]

Good evening @raj-prince , I have tried using the gcloud auth application-default login approach, among with manually setting the GOOGLE_APPLICATION_CREDENTIALS env var via os.environ["GOOGLE_APPLICATION_CREDENTIALS"] and manually writing the location in the CLI. My working environment is a Google Colab notebook, for some reason after performing this and other guides online, gcsfuse IS NOT ABLE to recognize the credentials. Any ideas?

Thanks for your time and work.

[Ayush Sethi]

@spolo96 Thanks for reaching out.

• Could you please confirm from where are you calling gcsfuse command ? Is that from terminal or from within Google colab notebook ? If it is from terminal, can you first confirm GOOGLE_APPLICATION_CREDENTIALS is set properly by running echo $GOOGLE_APPLICATION_CREDENTIALS from that terminal ?
• If the above is confirmed and still don't work, can you confirm that running gcsfuse command by passing the same path of JSON key file to --key-file flag of gcsfuse works ?

[Samuel Polo Peña]

@sethiay thanks a lot for the quick reply! I managed to make the command work in Colab thanks to your second point you stated, so if anyone also has this error, I highly advise to use the following command that worked for me:

!gcsfuse --key-file service-account.json --implicit-dirs "bucket-ml-test" "google_bucket_folder"

• --implicit-dirs copies every subdirectory from the bucket.
• bucket-ml-test is the name of my bucket in GCP.
• google_bucket_folder is a folder I previously created in my Colab environment to properly mount the data from bucket-ml-test.

P.D: @sethiay echo $GOOGLE_APPLICATION_CREDENTIALS returned the path of my service account JSON, but as I stated before, I managed to get it working with the previous command.