bake: default environment variable takes over override
asvinours opened this issue · comments
Contributing guidelines
- I've read the contributing guidelines and wholeheartedly agree
I've found a bug and checked that ...
- ... the documentation does not mention anything about my problem
- ... there are no open or closed issues that are related to my problem
Description
When defining a variable with a default value coming from the environment, bake will always take the value from the environment variable even if another bake file defines an override. The only way for the override to be used is for the environment variable to be entirely absent (setting it to an empty value is not enough)
Expected behaviour
The override should take precedence over the default value
Actual behaviour
the value from the environment variable is used
Buildx version
github.com/docker/buildx v0.14.0 171fcbe
Docker info
Client: Docker Engine - Community
Version: 26.0.0
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.14.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.25.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 8
Server Version: 26.0.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
runc version: v1.1.12-0-g51d5e94
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.5.0-1018-aws
Operating System: Ubuntu 22.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 30.64GiB
Name: ip-10-13-2-22
ID: 2180b260-4845-4a4a-9801-cf350854cac6
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Builders list
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default* docker
\_ default \_ default running v0.13.1 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
Configuration
# docker-bake-base.hcl
variable "IMAGE_NAME" {
default="$IMAGE_NAME"
}
variable "AWS_ACCOUNT_ID" {
default = "$AWS_ACCOUNT_ID"
}
variable "AWS_REGION" {
default = "$AWS_REGION"
}
target "default" {
output = [
"${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${IMAGE_NAME}",
]
}
# docker-bake.hcl
IMAGE_NAME="john"
target "my-target" {
inherits = ["default"]
}
IMAGE_NAME="bob" docker buildx bake -f docker-bake-base.hcl -f 'docker-bake.hcl' --print my-target
{
"group": {
"default": {
"targets": [
"my-target"
]
}
},
"target": {
"my-target": {
"context": ".",
"dockerfile": "Dockerfile",
"output": [
"$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/bob"
]
}
}
}
IMAGE_NAME="" docker buildx bake -f docker-bake-base.hcl -f 'docker-bake.hcl' --print my-target
{
"group": {
"default": {
"targets": [
"my-target"
]
}
},
"target": {
"my-target": {
"context": ".",
"dockerfile": "Dockerfile",
"output": [
"$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/"
]
}
}
}
docker buildx bake -f docker-bake-base.hcl -f 'docker-bake.hcl' --print my-target
{
"group": {
"default": {
"targets": [
"my-target"
]
}
},
"target": {
"my-target": {
"context": ".",
"dockerfile": "Dockerfile",
"output": [
"$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/john"
]
}
}
}
Build logs
No response
Additional info
No response
That looks like expected behavior with the correct order of precedence. Configuration passed in from env and flags overrides the config loaded from files. If you worry about collisions, you should define a more unique name, eg. BUILDX_MYAPP_IMAGE_NAME
. If you don't want your arg to be set from environment then don't define "variable" block and only define the attribute in the hcl definition.
I see. I think I see where the confusion came from for me: reading the use environment variable as default
documentation, I had understood that for an environment variable to be used, the bake variable default value had to be set to the name of an environment variable.
Doing more testing, I realised that I was wrong.
Setting the default value of a bake variable to an environment variable name doesn't seem to have much effect in itself.
What does, seem to be that the environment variable name has to match the bake variable name and the attribute is defined with a variable block:
variable "IMAGE_NAME" {
default="$MY_IMAGE_NAME"
}
target "default" {
output = [
"123456123456.dkr.ecr.us-east-1.amazonaws.com/${IMAGE_NAME}",
]
}
export MY_IMAGE_NAME="bob";
docker buildx bake -f docker-bake.hcl --print
{
"target": {
"default": {
"context": ".",
"dockerfile": "Dockerfile",
"output": [
"123456123456.dkr.ecr.us-east-1.amazonaws.com/$MY_IMAGE_NAME"
]
}
}
}
On the other hand:
variable "IMAGE_NAME" {
default="hardcoded-default-name"
}
target "default" {
output = [
"123456123456.dkr.ecr.us-east-1.amazonaws.com/${IMAGE_NAME}",
]
}
export IMAGE_NAME="bob";
docker buildx bake -f docker-bake-base.hcl --print
{
"target": {
"default": {
"context": ".",
"dockerfile": "Dockerfile",
"output": [
"123456123456.dkr.ecr.us-east-1.amazonaws.com/bob"
]
}
}
}
As you mentioned, to prevent bake from reading from the environment variables, I can define the attribute without the variable block.