Implement GoogleCloud storage backend
constantinpape opened this issue · comments
The new C++ API allows to implement other backends than the filesystem.
There is a mock-up implementation for this already.
In order to implement actually implement it, the functionality in the gcs namespace needs to be implemented using the google cloud sdk. For reference, see the implementation in namespace filesystem.
The steps involved should be analog to #136. I would prefer to have the aws implementation first, but if someone wants to take a shot at implementing the gcs binding this would be very welcome too.
Trying to build with GCS
/usr/local/google/home/vharron/dev/sca/z5/include/z5/gcs/dataset.hxx:101:25: error: cannot declare field 'z5::gcs::Dataset<signed char>::handle_' to be of abstract type 'z5::gcs::handle::Dataset'
101 | handle::Dataset handle_;
| ^~~~~~~
I'm trying to figure out what should be happening, so I'm trying to understand what handles do. I'm not sure what the purpose of handles are. It would be great to have an explanation comment around
include/z5/handle.hxx:12
I'm trying to figure out what should be happening, so I'm trying to understand what handles do. I'm not sure what the purpose of handles are. It would be great to have an explanation comment around
include/z5/handle.hxx:12
That's a good point, I will try to document this part of the C++ API better.
Please note that neither S3 nor GCS are fully supported yet; the API is defined but many of the functions are not implemented yet.
Any help on this would be highly appreciated!
@vharron
I have fixed the build issues with GCS now and added some explanation on the handles, see
https://github.com/constantinpape/z5/blob/master/include/z5/handle.hxx#L14.