googleCloudStorageR, gcs_save() works but gcs_load() seems to fail

[Jamie]

Hi, I've hit the same issue as described here on Stack overflow.

REPREX

googleCloudStorageR::gcs_save(
  iris,
  file = 'bucket-folder/iris.rda',
  bucket = 'our-gcs-bucket'
)
googleCloudStorageR::gcs_load(
  file = 'bucket-folder/iris.rda',
  bucket = 'our-gcs-bucket'
)

returns error messages.

Error in curl::curl_fetch_disk(url, x$path, handle = handle): Failed to open file C:\Users\myname\path-to-file\iris.rda.
Request failed [ERROR]. Retrying in 1 seconds...
Error in curl::curl_fetch_disk(url, x$path, handle = handle): Failed to open file C:\Users\myname\path-to-file\iris.rda.
Request failed [ERROR]. Retrying in 1.3 seconds...
Error in curl::curl_fetch_disk(url, x$path, handle = handle) : 

https://stackoverflow.com/questions/68087081/googlecloudstorager-gcs-save-works-but-gcs-load-does-not

Any help or advise you could offer would be really appreciated, Thanks!

[Mark Edmondson]

I think you will become unstuck if you use gcs_get_object() instead of gcs_load() which is what the gcs_load() uses underneath.

As to the underlying issue, will take a look. I think it may be that you need to create the folder locally? e.g. bucket-folder/

[Mark Edmondson]

Yes thats it:

googleCloudStorageR::gcs_save(
  iris,
  file = 'bucket-folder/iris.rda',
  bucket = 'our-gcs-bucket'
)
dir.create("bucket-folder")
googleCloudStorageR::gcs_load(
  file = 'bucket-folder/iris.rda',
  bucket = 'our-gcs-bucket'
)
✓ Saved bucket-folder/iris.rda to bucket-folder/iris.rda  ( 1.1 Kb )
[1] TRUE

The name you give the file to the GCS bucket is just a name with a / in it so it succeeds, but when downloading again its a file structure that fails if not present.

[Mark Edmondson]

You could alternatively specify the file location with no folder name:

googleCloudStorageR::gcs_load(
  file = 'bucket-folder/iris.rda',
  saveToDisk = "iris.rds",
  bucket = 'our-gcs-bucket'
)