chrislusf / gleam

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support for reading multiple files from S3

JensRantil opened this issue · comments

I notice that the S3 virtual file system doesn't support listings. I also don't find any example code that shows how to use multiple S3 files as input so I assume it's currently not supported. I think that's a pretty common use-case so I'm creating a feature request here. ✋

I guess the same issue applies to the Google Storage VFS. 🤷

Finally, is there a workaround? I guess I could write my custom func(io.Writer, *pb.InstructionStat) error, perhaps? I'm new to this library.