Container Instances should support environment variable names containing dot '.'
stephanwehr opened this issue · comments
System Info
OS: Windows 10
Shell: Ubuntu 20.04 on Windows
Azure CLI: 2.36.0
Expected Behavior
Executing the following Azure CLI command should create a container instance running Elasticsearch:
az container create --image elasticsearch:7.3.2 --name es -g test-rg --memory 2 --cpu 1 --ports 9200 --environment-variables "discovery.type=single-node"
Using environment variable names containing dot '.' should work.
Current Behavior
Executing the Azure CLI command exit with an error:
(InvalidContainerEnvironmentVariable) The environment variable name in container 'es' of container group 'es' is invalid. A valid environment variable name must start with alphabetic character or '_', followed by a string of alphanumeric characters or '_' (e.g. 'my_name', or 'MY_NAME', or 'MyName').
Code: InvalidContainerEnvironmentVariable
Message: The environment variable name in container 'es' of container group 'es' is invalid. A valid environment variable name must start with alphabetic character or '_', followed by a string of alphanumeric characters or '_' (e.g. 'my_name', or 'MY_NAME', or 'MyName').
Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @macolso.
Issue Details
System Info
OS: Windows 10
Shell: Ubuntu 20.04 on Windows
Azure CLI: 2.36.0
Expected Behavior
Executing the following Azure CLI command should create a container instance running Elasticsearch:
az container create --image elasticsearch:7.3.2 --name es -g test-rg --memory 2 --cpu 1 --ports 9200 --environment-variables "discovery.type=single-node"
Using environment variable names containing dot '.' should work.
Current Behavior
Executing the Azure CLI command exit with an error:
(InvalidContainerEnvironmentVariable) The environment variable name in container 'es' of container group 'es' is invalid. A valid environment variable name must start with alphabetic character or '_', followed by a string of alphanumeric characters or '_' (e.g. 'my_name', or 'MY_NAME', or 'MyName').
Code: InvalidContainerEnvironmentVariable
Message: The environment variable name in container 'es' of container group 'es' is invalid. A valid environment variable name must start with alphabetic character or '_', followed by a string of alphanumeric characters or '_' (e.g. 'my_name', or 'MY_NAME', or 'MyName').
| Author: | stephanwehr |
|---|---|
| Assignees: | - |
| Labels: |
|
| Milestone: | - |
@stephanwehr Thanks for reaching out to us and sharing this feedback. I was able to reproduce this issue from AzCLI as well as Powershell. Post isolating this issue, I found that this issue isn't related to the Container Instance but the underlying image OS type.
The bash manual mentions this clearly (here)
name
A word consisting solely of letters, numbers, and underscores, and beginning with a letter or underscore. Names are used as shell variable and function names. Also referred to as an identifier.
The elasticSearch image seems to be using Linux OSType. So the fix / workaround has to come from the OS side and not from the AzCLI / REST API Spec. Hope this helps.
@navba-MSFT you are correct, Bash can't but Elasticsearch can.
With the "env" command it is possible to set environment variables containing special characters.
Here is an example:
# env -i "discovery.type=single-node" "my:test:var=value" /bin/bash -c "env"
my:test:var=value
discovery.type=single-node
PWD=/root
SHLVL=0
_=/usr/bin/env
In Docker compose the environment can be set like this:
version: '2.2'
services:
elasticsearch:
image: elasticsearch:7.3.2
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- discovery.type=single-node
ports:
- 9200:9200
I've dug a little bit deeper on this issues and it seems Linux executes bash and other programs with execve().
How Linux execute programs can be seen using "strace":
$ strace env -i "my:test:var=some text" /bin/bash -c ":" 2>&1 | grep "execve"
execve("/usr/bin/env", ["env", "-i", "my:test:var=some text", "/bin/bash", "-c", ":"], 0x7ffd31dcb018 /* 21 vars */) = 0
execve("/bin/bash", ["/bin/bash", "-c", ":"], 0x5577895c20f0 /* 1 var */) = 0
As I understand, on Linux container environments like Docker for example set the initial environment before executing the 1st program inside the container. There is a process environment and bash environment which both can be retrieved using commands like "env" or "printenv".
I assume referring to this error message that the API performs a syntax check on variable names which should not be there:
{"error":{"code":"InvalidContainerEnvironmentVariable","message":"The environment variable name in container 'es' of container group 'es' is invalid. A valid environment variable name must start with alphabetic character or '_', followed by a string of alphanumeric characters or '_' (e.g. 'my_name', or 'MY_NAME', or 'MyName')."}}
When a program is executed, a file containing the initial environment is created on the proc filesystem.
/proc/[pid]/environ
This file contains the initial environment that was set
when the currently executing program was started via
execve(2). The entries are separated by null bytes
('\0'), and there may be a null byte at the end. Thus, to
print out the environment of process 1, you would do:
$ cat /proc/1/environ | tr '\000' '\n'
If, after an execve(2), the process modifies its
environment (e.g., by calling functions such as putenv(3)
or modifying the environ(7) variable directly), this file
will not reflect those changes.
Furthermore, a process may change the memory location that
this file refers via prctl(2) operations such as
PR_SET_MM_ENV_START.
Permission to access this file is governed by a ptrace
access mode PTRACE_MODE_READ_FSCREDS check; see ptrace(2).
@stephanwehr I just wanted to confirm if you are able to get this working with elasticsearch image with Linux OStype ? Awaiting your reply.
@navba-MSFT I'm not able to get Elasticsearch to work with Linux OStype because environment variable names containing dot '.' are not supported.
@navba-MSFT referring to your last question:
@stephanwehr I just wanted to confirm if you are able to get this working with elasticsearch image with Linux OStype ? Awaiting your reply.
This issue is not primarily about if Elasticsearch is working with container instances or not.
Elasticsearch is an example where environment variable names containing dot are used.
Please see this issue as a feature request to support environment variable names containing dot.
@stephanwehr AFAIK, this ask is something which violates the IEEE standard for environment variables.
Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore)
Refer this.
@stephanwehr AFAIK, this ask is something which violates the IEEE standard for environment variables.
Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore)
Refer this.
@navba-MSFT as stated this applies to the Shell and Utilities and not necessarily to other programs like Elasticsearch.
There is no hard requirement under Linux or inside a Docker container to use a shell to execute programs.
Linux itself allows variable names of type char:
#include <stdlib.h>
int setenv(const char *name, const char *value, int overwrite);
int unsetenv(const char *name);@stephanwehr Thanks for clarifying. I have added the Service Team to look into this feature request ask.
Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!
Hi, @stephanwehr. The PR will be closed since the PR has no update for 28 days. If you still need the PR review to proceed, please reopen it and @ mention PR assignee.
I've tried hosting ElasticSearch on ACI and got this issue.
Are there any updates?
Same for me tried to use ElasticSearch with ACI and I get the same error with dotted env variable names. Why is this closed? There is no fix or workaround
ACI and Elasticsearch and same issue 👍