Support for env var substitutions in env.str and env.dict and other composite types
gnarvaja opened this issue · comments
I found some use cases where doing env var substitution inside the VALUE of another envvar it's very helpful.
For example:
DB_URI=postgres://${PGUSER:-gnarvaja}:${PGPASS}@myserver/mydb
PGPASS=awesome
And being able to read the variable like this would be awesome:
>>> env.str("DB_URI", substitute_envs=True)
postgres://gnarvaja:awesome@myserver/mydb
The main use case is when you have the user/pass as individuals secrets (perhaps handled by an operator like https://github.com/zalando/postgres-operator) but you need a URI with parts that don't need to be secret.
Another use case is a workaround when composite variables like dict become too complex. For example:
INIT_KARGS="name=foo,script_params=--http-header Referer=https://radiocut.fm"
If you do env.dict("INIT_KARGS")
you get ValueError: too many values to unpack
because of the "=" in the second value. So in this case you can do:
INIT_KARGS="name=foo,script_params=--http-header ${HTTP_HEADER}"
HTTP_HEADER="Referer=https://radiocut.fm"
>>> env.dict("INIT_KARGS", substitute_envs=True)
{"name": "foo", "script_params": "--http-header Referer=https://radiocut.fm"}
We can use a syntax like this for the references to the environments variables: ${VARIABLE[:-default][:ttype]}. For example ${YEAR:-2020:tint}
. (the :- is how you specify defaults in bash, the third :t parameter is for specifying the type is not str).
Here is the code I'm using in my project (a bit ugly):
envvar_matcher = re.compile(r'\$\{([A-Za-z0-9_]+)(:-[^\}:]*)?(:t[^\}]*)?\}')
def _replace_envs(value):
ret = ""
prev_start = 0
for m in envvar_matcher.finditer(value):
env_name = m.group(1)
env_default = m.group(2)
env_type = m.group(3)
if env_type is None:
method = env.str
else:
method_name = env_type[2:]
method = getattr(env, method_name)
if env_default is None:
env_value = method(env_name)
else:
env_default = env_default[2:]
env_value = method(env_name, default=env_default)
if m.start() == 0 and m.end() == len(value):
return env_value
ret += value[prev_start:m.start()] + env_value
prev_start = m.end()
ret += value[prev_start:]
return ret
If you agree with this new feature, I can try to do a PR. The new feature will be activated only if explicitly using substitute_envs=True
as parameter.
Reviewing the docs and the open PRs, perhaps this feature can be merged or replace Proxied Variables. Proxied variables are a special case of this substitution.
Anyway, I think it shouldn't be enabled by default, as this can be a security issue (For example someone sets: TITLE=${SECRET_ENV_VAR}
)
Interesting idea. I'm not opposed to this in principle, esp if it doesn't add too much complexity. I agree that it could possibly replace proxied variables.
Would you like to put up a PR sketch?
Done, check #168 .
I wasn't sure about proxied variables, perhaps a path for deprecation might be raising DeprecationWarning for a few versions and later remove
Great, thanks! I won't be able to review the PR in depth for the next few days--work and life commitments all coming fast--but I'll do my best not to lose track of it.