ArroyoSystems / arroyo

Distributed stream processing engine in Rust

Home Page:https://arroyo.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Validate StorageBackend on Service Startup

jacksonrnewhouse opened this issue · comments

Both arroyo-controller and arroyo-compiler-service need to connect to the configured storage backend in order to run correctly. However, right now they don't attempt a connection until a user tries to run a pipeline, which makes it hard to tell if the configuration is correct. It'd be great to have them test the connection. I think it would be sufficient to make a get() call on the underlying ObjectStore and confirm it returns either the result or object_store::Error::NotFound.

If it is unable to connect it should fail quickly, relying on orchestration to revive it.