Starting scheduler with docker-compose on `main` throws error: Error: An unknown argument '--config-backend' was specified.

[Paolo Rechia]

Describe the bug
Starting the deployment in the main branch with the docker-compose option ttriggers this error:

arrow-ballista-ballista-scheduler-1  | Starting nginx to serve Ballista Scheduler web UI on port 80
arrow-ballista-ballista-scheduler-1  | Error: An unknown argument '--config-backend' was specified.

To Reproduce
Execute docker-compose up --build from main branch.

Expected behavior
Expected services to execute successfully.

Additional context
When looking at the source code, specifically ballista/scheduler/src/bin/main.rs I could not find the relevant etcd options.

Changing the docker-compose.yml line 29 from

    command: "--config-backend etcd --etcd-urls etcd:2379 --bind-host 0.0.0.0"

To

    command: " --bind-host 0.0.0.0"

Fixes the error and deploys successfully the containers. It does not seem like a proper solution here, as we're not effectively using the etcd service. Is there an unfinished feature in main branch?

[Paolo Rechia]

I seem to have found the proper cause of the error. In the file ballista/scheduler/scheduler_config_spec.toml I found this block:

[[param]]
abbr = "b"
name = "cluster_backend"
type = "ballista_scheduler::cluster::ClusterStorage"
doc = "The configuration backend for the scheduler cluster state, possible values: etcd, memory, sled. Default: sled"
default = "ballista_scheduler::cluster::ClusterStorage::Sled"

Thus changing the config in docker-compose to:

    command: "--cluster-backend etcd --etcd-urls etcd:2379 --bind-host 0.0.0.0"

Seems to work. I might create a follow-up PR here.