{brickster} is the R toolkit for Databricks, it includes:
-
Wrappers for Databricks API's (e.g.
db_cluster_list,db_volume_read) -
Browser workspace assets via RStudio Connections Pane (
open_workspace()) -
Exposes the
databricks-sql-connectorvia{reticulate}(docs) -
Utility functions to streamline workloads in Databricks notebooks
remotes::install_github("databrickslabs/brickster")
library(brickster)
# requires `DATABRICKS_HOST` only if using OAuth U2M
# first request will open browser window to login
Sys.setenv(DATABRICKS_HOST = "<workspace-prefix>.cloud.databricks.com")
# list all SQL warehouses
warehouses <- db_sql_warehouse_list()
# read `data.csv` from a volume
file <- db_volume_read(
path = "/Volumes/<catalog>/<schema>/<volume>/data.csv",
tempfile(pattern = ".csv")
)
volume_csv <- readr::read_csv(file)Refer to the "Connect to a Databricks Workspace" article for more details on getting authentication configured.
{brickster} is very deliberate with choosing what API's are wrapped. {brickster} isn't intended to replace IaC tooling (e.g. Terraform) or to be used for account/workspace administration.
| API | Available | Version |
|---|---|---|
| DBFS | Yes | 2.0 |
| Secrets | Yes | 2.0 |
| Repos | Yes | 2.0 |
| mlflow Model Registry | Yes | 2.0 |
| Clusters | Yes | 2.0 |
| Libraries | Yes | 2.0 |
| Workspace | Yes | 2.0 |
| Endpoints | Yes | 2.0 |
| Query History | Yes | 2.0 |
| Jobs | Yes | 2.1 |
| Volumes (Files) | Yes | 2.0 |
| SQL Statement Execution | Yes | 2.0 |
| REST 1.2 Commands | Partially | 1.2 |
| Unity Catalog | Partially | 2.1 |