cloudyr / googleCloudStorageR

Google Cloud Storage API to R

Home Page:https://code.markedmondson.me/googleCloudStorageR

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Get Full size of object, in bytes

muschellij2 opened this issue · comments

I would like the size (in bytes) to check against a listing to make sure all bytes are downloaded. The issue is that googleCloudStorageR automatically formats this:

x$size <- vapply(as.numeric(x$size), function(x) format_object_size(x, "auto"), character(1))

And you can't set an option for format_object_size. I think if you use something like:

format_object_size(getOption("googleCloudStorageR.format_units", default = "auto"))

Then you can set an option("googleCloudStorageR.format_units" = "b") then this should work I think

head(googleCloudStorageR::gcs_list_objects()$size)
#> [1] "218.8 Mb"  "137.9 Mb"  "1.1 Kb"    "45.6 Kb"   "74 bytes"  "159 bytes"
head(googleCloudStorageR::gcs_list_objects(detail = "full")$size)
#> [1] "218.8 Mb"  "137.9 Mb"  "1.1 Kb"    "45.6 Kb"   "74 bytes"  "159 bytes"

Created on 2022-03-07 by the reprex package (v2.0.1)

I'd rather set fewer rather than more options, how about adding a column to the data.frame output called "size_bytes" ?

Yeah I think that'd work too. Just figured i on the one hand you have checks on the column names or something like that it'd be an issue. On the other hand, you may have data type checks which may be different (size is character, size_bytes is likely numeric).

Lets try it see what breaks ;) it may be a chance to also inspect what the API returns, it may have some added fields from when it was first done that aren't surfaced (perhaps versioning?)