support for path-style requests on s3.createMultipartUpload

[Fabricio C Zuardi]

AWS currently supports both path-style and virtual-hosted–style URLs and even though path-style is to be deprecated soon, other S3-compatible providers have incomplete support for virtual-hosted-style URLs. See https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access

On the lib code, the usage is mixed, on methods like putObject it uses path-style while on others like createMultipartUpload it uses virtual-hosted-style.

Amazon CLI when provided with a --endpoint argument uses path-style for thei own requests 🤣 for example:

aws s3api --endpoint <endpoint url> create-multipart-upload --bucket <bucket name> --key <object key> --debug

shows:

...
POST
/<bucket name>/<object key>
uploads=
...

Should this be a config of the S3Client? Should we do path-styleby default on createMultipartUpload and follow aws-cli implementation?

[Théo Crevon]

Hey @fczuardi 👋🏻

Thanks for bringing this to our attention 🙇🏻

The library has received many contributions over time, and there are likely inconsistencies in our implementation indeed. Especially we received contributions to make it possible to use the library against third-party S3-compatible providers such as Ceph, Cloudflare, and DigitalOcean spaces.

I must admit that I implemented the API support based on what was officially documented, and this question of path vs hostname has been coming back and forth since I started this project. AWS APIs are anything but consistent, and if my memory doesn't fail me, we implemented support for path-based only to cater to certain specific providers.

I will have to do a bunch of research to see what makes the most sense to remain somewhat compatible with third-party S3-compatible hosts (which we do not officially support, but as long as it's a low effort to make the library compatible, are willing to cater to).