abs51295 / fabric8-analytics-scaler

Service that scales workers based on SQS queue size

fabric8-analytics-worker-scaler

OpenShift service that scales fabric8-analytics workers based on number of messages in SQS. This service directly uses oc command-line tool to scale the deployment. This requires to have mounted service account inside the container. workerscaler service account for that purpose by default. Definitions and role binding of the service account are available in the openshift/ directory.

Configuration

The service needs to be configured via environment variables. The list of variables recognized by the service follows.

DC_NAME

Name of the deployment config to scale. Default value: bayesian-worker-ingestion

DEFAULT_REPLICAS

Default (minimum) number of replicas. Default value: 1

MAX_REPLICAS

Max number of replicas. Default value: 2

SQS_QUEUE_NAME

Name of the queue to monitor (without deployment prefix). Default value: ingestion_bayesianFlow_v0,ingestion_bayesianPackageFlow_v0

OC_PROJECT

Name of the project in OpenShift where to apply the changes. Default value: bayesian-preview

SCALE_COEF

Number of messages that singe worker is able to process, i.e. no need to spawn additional pods. Default value: 100

SLEEP_INTERVAL

Interval, in minutes, how often to run the scaler. Default value: 5

DRY_RUN

Run in dry-run mode, do not scale anything. Default value: false

The default values can be tweaked directly in the template. The values for specific deployments (staging, production) can be set in openshiftio/saas-analytics repository.

Deployment on openshift

OpenShift deployment information is located at openshift directory.