Databricks Admin Center (DAC)

Databricks Admin Center (DAC) solves the problem of cost- and performance monitoring, maintenance and overview of Databricks workspaces, clusters and users in one place.

Configuration

DAC has two configuration files:

.env

export FLASK_APP=dac.py
export FLASK_ENV=production
export DAC_CONFIG_PATH=configs/config.json

configs/config.json

{
    "workspaces": [
        {
            "url": "westeurope.azuredatabricks.net/?o=[workspace_id]",
            "id": "[workspace_id]",
            "type": "[AZURE|AWS]",
            "name": "[workspace_name]",
            "token": "[token]"
          }, ...
    ],
    "prices": [
        {
          "type": "interactive",
          "value": 1.0
          },
        {
          "type": "job",
          "value": 1.0
        }
    ],
    "thresholds": [
    ],
    "scraper": {
        "interval": 270
    }
}

Install

pip install -r requirements.txt

We recommend using virtualenv to run DAC. DAC requires Python 3.6+.

Running

DAC can be run in a docker container or without it.

Non-containerized running

To run the UI

flask run

To run the scraper:

# Only for the first run:
python main.py create_db

# To run the scraper
python main.py scrape

Dockerized running

1. Create the config files

2. Build image

./scripts/create-docker

3. Run image

./scripts/run-local

Development

If you want to run Flask in development mode, change the following line in .env:

export FLASK_ENV=production

export FLASK_ENV=development

Design doc and contributing

Main features, major architecture or code changes must be designed and planned in a design document. These documents are available on the DAC wiki. You are required to provide a design document if the feature you would like to contribute requires one. A minimal design doc template can be used from the wiki.

Support

The main channel for DAC is the official DAC Slack, where you can ask any question. Feel free to join us!

Slack: project-dac.slack.com

Roadmap

DAC Roadmap is available in the wiki: Roadmap

The roadmap is subject to change. If you feel excited about certain features please let us know. Even better, if you consider contributing features feel free to contact the development team for any support you need.

Sponsoring

DAC is currently developped with ❤️ at Datapao. If you need a feature or you think DAC is useful, please consider sponsoring the development.

mikepetridisz / dac