metadata should be generated before calling `Store` implementation Upload/Retrieve
discentem opened this issue · comments
Right now each Store (S3, GCS, Blob, and any plugin) implementation is required to generate .cfile metadata themselves and write it to the file system. internal/metadata provides various helper functions for generating metadata files (.cfiles) but still each implementation must implement this. This shouldn't be necessary and doesn't really fit the spirit of the framework.
Ideally, Store implementations should perform only upload and retrieval operations after metadata is already generated. If necessary Store implementations could call back to the code in internal/cli/upload.go to cancel/delete metadata generation.