Using build_in_docker running docker in docker
reikje opened this issue Β· comments
Description
I am struggling using the build_in_docker
option when running inside a VSCode devcontainer.
I arrived here because my Lambda needs a Python dependency called cryptography
and I am hitting /lib64/libc.so.6: version 'GLIBC_2.28' not found
when I run terraform apply
on my Macbook.
My project is running inside a .devcontainer
and I am mounting "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
into that container. When I then run terraform apply
inside the container, it basically is running docker in docker utilising Docker Desktop on the host. Using the build_in_docker
option will result in the following error:
21826ea7c95e: Pull complete
β Digest: sha256:79efc767616c002acf83758a24da5fb02d3d079313321b0de304bbfec402b77d
β Status: Downloaded newer image for public.ecr.aws/sam/build-python3.10:latest
β docker: Error response from daemon: Mounts denied:
β The path /root/.ssh/known_hosts is not shared from the host and is not known to Docker.
Here is the mount causing the issue. I am really not a docker expert but I think instead of using /root
(which is $HOME inside my devcontainer) I need to use the path actual available on the host. There seem to be no option to set the folder for the .ssh
directory ...
- [ x ] β I have searched the open/closed issues and my issue is not listed.
β οΈ Note
Before you submit an issue, please perform the following first:
- Remove the local
.terraform
directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!):rm -rf .terraform/
- Re-initialize the project root to pull down modules:
terraform init
- Re-attempt your terraform plan or apply and check if the issue still persists
Versions
- Module version [Required]:
6.0.1 - Terraform version:
1.6.1 - Provider version(s):
5.21.0
Reproduction Code [Required]
.devcontainer/devcontainer.json
:
{
"name": "Python 3",
"build": {
"dockerfile": "Dockerfile",
"context": "..",
"args": {
}
},
"customizations": {
"vscode": {
"settings": {
"editor.suggest.insertMode": "replace",
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true,
"python.defaultInterpreterPath": "/var/folders/.venv/bin/python",
"python.analysis.indexing": true,
"python.analysis.packageIndexDepths": [
{ "name": "", "depth": 2, "includeAllSymbols": true }
]
},
"extensions": [
"ms-python.python",
"charliermarsh.ruff",
"hashicorp.terraform",
"njpwerner.autodocstring"
]
}
},
"remoteUser": "root",
"runArgs": ["--add-host=host.docker.internal:host-gateway"],
"containerEnv": {
"GITHUB_WORKSPACE": "${localWorkspaceFolder}"
},
"mounts": [
"source=${env:HOME}${env:USERPROFILE}/.aws,target=/root/.aws,type=bind,readonly",
"source=${env:HOME}${env:USERPROFILE}/.ssh,target=/root/.ssh,type=bind,readonly",
"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
],
"features": {
"github-cli": "latest"
},
"workspaceMount": "source=${localWorkspaceFolder},target=/var/folders,type=bind",
"workspaceFolder": "/var/folders"
}
lambda.tf
:
module "lambda_function" {
source = "terraform-aws-modules/lambda/aws"
version = "~> 6.0"
timeout = 300
source_path = "../../src"
function_name = "whatever"
handler = "app.handler"
runtime = "python3.10"
create_sam_metadata = true
publish = true
build_in_docker = true
allowed_triggers = {
AnyRule = {
principal = "events.amazonaws.com"
source_arn = "arn:aws:events:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:rule/*"
}
}
tags = local.tags
}
Steps to reproduce the behavior:
- create a VSCode devcontainer
- open the project in the devcontainer
- run
terraform apply
in a module that has thebuild_in_docker
option set totrue
Expected behavior
The build runs properly inside a docker (in docker) container.
Actual behavior
The build fails with:
...
β 21826ea7c95e: Pull complete
β Digest: sha256:79efc767616c002acf83758a24da5fb02d3d079313321b0de304bbfec402b77d
β Status: Downloaded newer image for public.ecr.aws/sam/build-python3.10:latest
β docker: Error response from daemon: Mounts denied:
β The path /root/.ssh/known_hosts is not shared from the host and is not known to Docker.
.. unless the host system has a /root/.ssh
folder, which is not the case in MacOS.
I'm also looking at the similar case i.e. trying to use terraform-aws-lambda -module within dockerized build container.
I think mounting .ssh/known_hosts is necessary only if accessing packages in private repos via ssh. So basically it's possibly to use local version of terraform-aws-lambda-repository and modify package.py docker_run_command-function a bit. I commented out this part as a temp workaround. And of course one need to refer then to local path of terraform-aws-lambda directory.
# docker_cmd.extend([
# # '-v', '{}/.ssh/id_rsa:/root/.ssh/id_rsa:z'.format(home),
# '-v', '{}/.ssh/known_hosts:/root/.ssh/known_hosts:z'.format(home),
# ])
But then I'm facing another issue when install npm packages:
Error: local-exec provisioner error
β
β with module.nodejs_lambda_layer.null_resource.archive[0],
β on modules/terraform-aws-lambda/package.tf line 67, in resource "null_resource" "archive":
β 67: provisioner "local-exec" {
β
β Error running command
β './builds/lambda_layer/9b33c113e714cbd7294f9da60e90d6c4ca1f8babd889b16ef17adc9f073a6fe6.plan.json':
β exit status 1. Output: zip: creating
β './builds/lambda_layer/9b33c113e714cbd7294f9da60e90d6c4ca1f8babd889b16ef17adc9f073a6fe6.zip'
β archive
β > docker images '--format={{.ID}}' ecs-test-pipeline-infra-nodejs18.x-build
β Installing npm requirements: ./../nodejs-lambda/package.json
β > mktemp -d terraform-aws-lambda-XXXXXXXX #
β /tmp/terraform-aws-lambda-ljdocj4k
β > cd /tmp/terraform-aws-lambda-ljdocj4k
β > docker run --rm -w /var/task -v
β /tmp/terraform-aws-lambda-ljdocj4k:/var/task:z --entrypoint '' 8a0a80350522
β /bin/sh -c 'npm install && chown -R 0:0 .'
β npm ERR! code ENOENT
β npm ERR! syscall open
β npm ERR! path /var/task/package.json
β npm ERR! errno -2
β npm ERR! enoent ENOENT: no such file or directory, open
β '/var/task/package.json'
β npm ERR! enoent This is related to npm not being able to find a file.
β npm ERR! enoent
This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days
This issue was automatically closed because of stale in 10 days
I'm going to lock this issue because it has been closed for 30 days β³. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.