ArroyoSystems / arroyo

Distributed stream processing engine in Rust

Home Page:https://arroyo.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Invalid URL like s3::https://my-custom-s3/my-bucket/key/path

huaxk opened this issue · comments

When starting Worker when aws_endpoint is set like s3::https://my-custom-s3/my-bucket/key/path, throws error:
thread 'main' panicked at copy-artifacts/src/main.rs:31:26: Failed to download s3::https://my-custom-s3/my-bucket/key/path/pipeline: InvalidUrl

It should be that the aws_endpoint url without port cannot pass the verification.

The S3_ENDPOINT_URL regular expression:
^[sS]3[aA]?::(?<protocol>https?)://(?P<endpoint>[^:/]+):(?<port>\d+)/(?P<bucket>[a-z0-9\-\.]+)(/(?P<key>.+))?$
should be changed to:
^[sS]3[aA]?::(?<protocol>https?)://(?P<endpoint>[^:/]+)(:(?<port>\d+))?/(?P<bucket>[a-z0-9\-\.]+)(/(?P<key>.+))?$
to support this form of aws_endpoint url.