Using environment variables for connection string to MySQL db (running in docker)?
dev-samples opened this issue · comments
Is it possible to read e.g. hostname, user and password for a connection to e.g. and external MySQL db?
I have created this data-sources.json file
{
"folders": {},
"connections": {
"mysql8-18f4fad411e-67d520a52b28e036": {
"provider": "mysql",
"driver": "mysql8",
"name": "MySQL@${mysql-host.name}:3306",
"configuration": {
"host":"${mysql-host.name}",
"port": "33306",
"url": "jdbc:mysql://${mysql-host.name}:3306/",
"configurationType": "MANUAL",
"type": "dev",
"closeIdleConnection": true,
"auth-model": "native"
}
}
},
"connection-types": {
"dev": {
"name": "Development",
"color": "255,255,255",
"description": "Regular development database",
"auto-commit": true,
"confirm-execute": false,
"confirm-data-change": false,
"smart-commit": false,
"smart-commit-recover": false,
"auto-close-transactions": true,
"close-transactions-period": 1800,
"auto-close-connections": true,
"close-connections-period": 14400
}
}
}
but when I pass that to my docker run command it just gets ignored:
docker run \
--rm -it \
-e CB_SERVER_NAME='cloudbeaver'\
-e CB_ADMIN_NAME='administrator' \
-e CB_ADMIN_PASSWORD='S0mePazzworD' \
-e mysql-host.name='test-mysql' \
-p 8080:8978 \
--name dbbeaver \
dbbeaver
I found some info here:
https://github.com/bobbyiliev/dbeaver/wiki/Admin-Manage-Connections#using-environment-variables
but its not really clear how to do this - e.g. if there are only a set of fields that support this?
Hi @dev-samples
Yes, it is possible.
But using some environment variables is disabled by default.
You can enable it in .cloudbeaver.runtime
systemVariablesResolvingEnabled: "true"
We will update wiki documentation, thank you for bringing this to our attention! 🦫
This article could be helpful for you Server configuration
@EvgeniaBzzz first thanks for getting back so quickly! ❤️
I did take a look at:
https://github.com/dbeaver/cloudbeaver/wiki/Server-configuration
but in the sample provided in the server.database scope I assume thats for the built-in db used by the dbeaver webapp and not for the external db connections (data-sources)?
At the bottom of that page I can the go to this link for data-sources:
https://github.com/dbeaver/cloudbeaver/wiki/Configuring-server-datasources
that describes the data-sources.json. This file I have tried to add my external MySQL connection to and it actually works when I start up a fresh container. But its the values (hostname etc.) in there that I want to set based on environments variables I can't seem to get that to work.
I have:
FROM dbeaver/cloudbeaver:24.0.4
RUN mkdir /opt/cloudbeaver/workspace/.data
COPY .cloudbeaver.runtime.conf /opt/cloudbeaver/workspace/.data/.cloudbeaver.runtime.conf
COPY data-sources.json /opt/cloudbeaver/workspace/GlobalConfiguration/.dbeaver/data-sources.json
Where I have updated .cloudbeaver.runtime.conf
app.systemVariablesResolvingEnabled: "${CLOUDBEAVER_SYSTEM_VARIABLES_RESOLVING_ENABLED:false}",
to:
app.systemVariablesResolvingEnabled: "${CLOUDBEAVER_SYSTEM_VARIABLES_RESOLVING_ENABLED:true}",
and then the docker run and data-sources.json as described in:
#2591 (comment)
What am I missing?
Your config looks correct.
Did you set systemVariablesResolvingEnabled to true?
Also keep in mind that cloudbeaver.runtime.conf has a higher priority than cloudbeaver.conf.
Yes and when I start the app I do see the following values:
/opt/cloudbeaver/workspace/.data/.cloudbeaver.runtime.conf
"systemVariablesResolvingEnabled": true,
In:
root@4f5d975fcd14:/opt/cloudbeaver/conf# cat /opt/cloudbeaver/workspace/.data/.cloudbeaver.runtime.conf
{
"server": {
"serverName": "cloudbeaver",
"expireSessionAfterPeriod": 604800000,
"productSettings": {
"core.localization.localization": "en",
"core.theming.theme": "light",
"plugin.log-viewer.disabled": false,
"plugin.log-viewer.logBatchSize": 1000.0,
"plugin.log-viewer.maxFailedRequests": 3.0,
"plugin.log-viewer.maxLogRecords": 2000.0,
"plugin.log-viewer.refreshTimeout": 3000.0,
"plugin.sql-editor.autoSave": true,
"plugin.sql-editor.disabled": false,
"plugin.sql-editor.maxFileSize": 10240.0,
"sql.proposals.insert.table.alias": "PLAIN"
},
"database": {
"driver": "h2_embedded_v2",
"url": "jdbc:h2:/opt/cloudbeaver/workspace/.data/cb.h2.dat",
"initialDataConfiguration": "conf/initial-data.conf",
"backupEnabled": "${CLOUDBEAVER_DB_BACKUP_ENABLED:true}",
"pool": {
"minIdleConnections": 4.0,
"maxIdleConnections": 10.0,
"maxConnections": 100.0,
"validationQuery": "SELECT 1"
}
},
"sm": {
"passwordPolicy": {
"minLength": "${CLOUDBEAVER_POLICY_MIN_LENGTH:8.0}",
"minNumberCount": "${CLOUDBEAVER_POLICY_MIN_NUMBER_COUNT:1.0}",
"minSymbolCount": "${CLOUDBEAVER_POLICY_MIN_SYMBOL_COUNT:0.0}",
"requireMixedCase": "${CLOUDBEAVER_POLICY_REQUIRE_MIXED_CASE:true}"
}
}
},
"app": {
"anonymousAccessEnabled": true,
"supportsCustomConnections": false,
"publicCredentialsSaveEnabled": true,
"adminCredentialsSaveEnabled": true,
"enableReverseProxyAuth": false,
"forwardProxy": false,
"linkExternalCredentialsWithUser": true,
"redirectOnFederatedAuth": false,
"resourceManagerEnabled": true,
"showReadOnlyConnectionInfo": false,
"grantConnectionsAccessToAnonymousTeam": false,
"systemVariablesResolvingEnabled": true,
"resourceQuotas": {
"dataExportFileSizeLimit": 1.0E7,
"resourceManagerFileSizeLimit": 500000.0,
"sqlMaxRunningQueries": 100.0,
"sqlResultSetRowsLimit": 100000.0,
"sqlResultSetMemoryLimit": 2000000.0,
"sqlTextPreviewMaxLength": 4096.0,
"sqlBinaryPreviewMaxLength": 261120.0
},
"defaultNavigatorSettings": {},
"enabledFeatures": [],
"enabledAuthProviders": [
"local"
],
"enabledDrivers": [],
"disabledDrivers": [
"sqlite:sqlite_jdbc",
"h2:h2_embedded",
"h2:h2_embedded_v2",
"clickhouse:yandex_clickhouse",
"generic:duckdb_jdbc"
]
}
}
As (expected?) I see the default values in:
/opt/cloudbeaver/conf/cloudbeaver.conf
root@4f5d975fcd14:/opt/cloudbeaver/conf# cat /opt/cloudbeaver/conf/cloudbeaver.conf
{
server: {
serverPort: 8978,
workspaceLocation: "workspace",
contentRoot: "web",
driversLocation: "drivers",
sslConfigurationPath:"${CLOUDBEAVER_SSL_CONF_PATH:workspace/.data/ssl-config.xml}",
rootURI: "/",
serviceURI: "/api/",
productSettings: {
# Global properties
core.theming.theme: 'light',
core.localization.localization: 'en',
plugin.sql-editor.autoSave: true,
plugin.sql-editor.disabled: false,
# max size of the file that can be uploaded to the editor (in kilobytes)
plugin.sql-editor.maxFileSize: 10240,
plugin.log-viewer.disabled: false,
plugin.log-viewer.logBatchSize: 1000,
plugin.log-viewer.maxFailedRequests: 3,
plugin.log-viewer.maxLogRecords: 2000,
plugin.log-viewer.refreshTimeout: 3000,
sql.proposals.insert.table.alias: PLAIN
},
expireSessionAfterPeriod: 1800000,
develMode: false,
enableSecurityManager: false,
sm: {
enableBruteForceProtection: "${CLOUDBEAVER_BRUTE_FORCE_PROTECTION_ENABLED:true}",
maxFailedLogin: "${CLOUDBEAVER_MAX_FAILED_LOGINS:10}",
minimumLoginTimeout: "${CLOUDBEAVER_MINIMUM_LOGIN_TIMEOUT:1}",
blockLoginPeriod: "${CLOUDBEAVER_BLOCK_PERIOD:300}",
passwordPolicy: {
minLength: "${CLOUDBEAVER_POLICY_MIN_LENGTH:8}",
requireMixedCase: "${CLOUDBEAVER_POLICY_REQUIRE_MIXED_CASE:true}",
minNumberCount: "${CLOUDBEAVER_POLICY_MIN_NUMBER_COUNT:1}",
minSymbolCount: "${CLOUDBEAVER_POLICY_MIN_SYMBOL_COUNT:0}"
}
},
database: {
driver: "h2_embedded_v2",
url: "jdbc:h2:${workspace}/.data/cb.h2v2.dat",
initialDataConfiguration: "conf/initial-data.conf",
pool: {
minIdleConnections: 4,
maxIdleConnections: 10,
maxConnections: 100,
validationQuery: "SELECT 1"
},
backupEnabled: "${CLOUDBEAVER_DB_BACKUP_ENABLED:true}"
}
},
app: {
anonymousAccessEnabled: true,
anonymousUserRole: "user",
defaultUserTeam: "user",
grantConnectionsAccessToAnonymousTeam: false,
supportsCustomConnections: false,
showReadOnlyConnectionInfo: false,
systemVariablesResolvingEnabled: "${CLOUDBEAVER_SYSTEM_VARIABLES_RESOLVING_ENABLED:false}",
forwardProxy: false,
publicCredentialsSaveEnabled: true,
adminCredentialsSaveEnabled: true,
resourceManagerEnabled: true,
resourceQuotas: {
dataExportFileSizeLimit: 10000000,
resourceManagerFileSizeLimit: 500000,
sqlMaxRunningQueries: 100,
sqlResultSetRowsLimit: 100000,
sqlResultSetMemoryLimit: 2000000,
sqlTextPreviewMaxLength: 4096,
sqlBinaryPreviewMaxLength: 261120
},
enabledAuthProviders: [
"local"
],
disabledDrivers: [
"sqlite:sqlite_jdbc",
"h2:h2_embedded",
"h2:h2_embedded_v2",
"clickhouse:yandex_clickhouse"
]
}
}
Hi @dev-samples
cloudbeaver.conf affects all workspaces, cloudbeaver.runtime.conf affects only current workspace. Configs will be the same on the initial run and then you could adjust cloudbeaver.runtime.conf to your needs. So that's expected that you see default values in cloudbeaver.conf.
What do you see when you press Test button for your connection?
You will not see any changes in UI for this variable, it's expected result. But you should connect successfully if host value in docker run is valid.
It works thanks! And same for mysql dbname, mysql username and password:
"mysql8-18f4fad411e-67d520a52b28e036": {
"provider": "mysql",
"driver": "mysql8",
"name": "MySQL@${mysql.hostname}:3306",
"save-password": true,
"configuration": {
"host": "${mysql.hostname}",
"port": "3306",
"database": "${mysql.dbname}",
"url": "jdbc:mysql://${mysql.hostname}:3306/",
"configurationType": "MANUAL",
"type": "dev",
"closeIdleConnection": true,
"auth-model": "native",
"user": "${mysql.username}",
"password": "${mysql.password}"
}
}
The only thing now missing is either figuring out how to disable the webapp login or automatically login when accessing the app:
CB_SERVER_NAME=CloudBeaver CE Server
CB_ADMIN_NAME=administrator
CB_ADMIN_PASSWORD=MySecretPassword234
I went with granting anonymous admin (app is secured within restricted network)
@dev-samples thanks for update and your interest in CloudBeaver! 🦫