Giters

microsoft / azure-pipelines-agent

Azure Pipelines Agent 🚀

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[BUG]: Documentation does not work for Alpine

nathanblair opened this issue · comments

commented

What happened?

The directions here

https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#create-and-build-the-dockerfile-1

do not create a usable containerized pipeline agent on an alpine system.

Instead, running the ./config.sh (specifically the underlying ./bin/Agent.Listener) generates a Segmentation Fault with no other information.

I've provided a Dockerfile that can be built (can even remove the secrets as its not strictly required for the bug to occur but they would need to be there for everything to configuration step to work) to reproduce the error. I happen to be using Docker for macOS on an M1 MBP (arm64) but have specifically targeted the linux/amd64 platform for both build and runtime using a docker compose file.

minimal reproducible Dockerfile 
ARG AGENT_PATH=/opt/agent
ARG USER=agent
ARG WORKDIR=/azp
ARG AZP_WORK=_work

FROM alpine:3 as base
ARG USER
ARG WORKDIR
ARG AZP_WORK
ARG AGENT_PATH
ENV AGENT_PATH="${AGENT_PATH}" TARGETARCH="linux-musl-x64"
RUN apk update --no-cache && apk upgrade --no-cache \
  && apk add --no-cache bash curl git jq icu-libs \
  && mkdir -p ${WORKDIR}/${AZP_WORK} \
  && adduser -D ${USER} && chown -R ${USER}:${USER} ${WORKDIR} \
  && mkdir -p ${AGENT_PATH} \
  && chown -R ${USER}:${USER} ${AGENT_PATH}
WORKDIR ${AGENT_PATH}

FROM base as install
ARG AGENT_VERSION=3.236.1
ADD --chown=${USER}:${USER} https://vstsagentpackage.azureedge.net/agent/${AGENT_VERSION}/vsts-agent-linux-musl-x64-${AGENT_VERSION}.tar.gz /tmp/agent.tar.gz
RUN tar xf /tmp/agent.tar.gz -C ${AGENT_PATH} && rm -rf /tmp/*
USER ${USER}

FROM install as configure
ARG AGENT_NAME
ARG AZP_URL
ARG AZP_POOL
# https://github.com/microsoft/azure-pipelines-agent/issues/4641
RUN \
  --mount=type=secret,mode=0444,id=ARM_TENANT_ID,required \
  --mount=type=secret,mode=0444,id=ARM_CLIENT_ID,required \
  --mount=type=secret,mode=0444,id=ARM_CLIENT_SECRET,required \
  ./config.sh --unattended \
  --agent "${AGENT_NAME}" \
  --auth sp \
  --tenant_id $(cat /run/secrets/ARM_TENANT_ID) \
  --clientid $(cat /run/secrets/ARM_CLIENT_ID) \
  --clientsecret $(cat /run/secrets/ARM_CLIENT_SECRET) \
  --url "${AZP_URL}" \
  --pool "${AZP_POOL}" \
  --work "${AZP_WORK}" \
  --replace \
  --acceptTeeEula \
  && chmod +x ./run.sh

FROM configure as run
WORKDIR ${WORKDIR}
ENTRYPOINT [ "${AGENT_PATH}./run-docker.sh" ]

Versions

3.236.1

Environment type (Please select at least one enviroment where you face this issue)

  • Self-Hosted
  • Microsoft Hosted
  • VMSS Pool
  • Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

alpine:3

Version controll system

No response

Relevant log output

No logs were generated
commented

On a whim I thought this may have something to do with Docker for Desktop macOS's setting

Use Rosetta for x86_64/amd64 emulation on Apple Silicon

My setting was enabled when I was getting the segmentation fault. Since there is no release for linux-musl-arm64 yet I have to virtualize amd64. Something in dotnet does not allow it to be run with virtualized with Rosetta on ARM64 Mac's as I've had similar issues with other dotnet projects before (please somebody rescue the world from dotnet).

Anyway, disabling Rosetta virtualization over amd64 made the segmentation fault go away.

What should be done is

  1. documentation that this setting should be disabled for Apple Silicon Docker for Desktop users
    1a) While we're on the documentation kick - we should also update the documentation for the alpine Dockerfile to note that you need to apk add bash as well.
  2. as a dotnet community we absolutely need to nail down what is causing these virtualization issues to occur for Rosetta. go/rust do not have this problem.
commented

Reopening because I didn't actually mean to close it as I'd like to get some official eyes and thoughts on this situation.

commented

Hi @nathanblair, thanks for the reporting! We have higher priority issues now, but we'll get back to this one soon

commented

Hi @nathanblair

Agent for Alpine ARM64 is released. Feel free to use it.

Docker file azp-agent-arm64v8-alpine.dockerfile —

FROM arm64v8/alpine
ENV TARGETARCH="linux-musl-arm64"

RUN apk update
RUN apk upgrade
RUN apk add bash curl git icu-libs jq

WORKDIR /azp/

COPY ./start.sh ./
RUN chmod +x ./start.sh

RUN adduser -D agent
RUN chown agent ./
USER agent
# Another option is to run the agent as root.
# ENV AGENT_ALLOW_RUNASROOT="true"

ENTRYPOINT ./start.sh

Build the docker image —

docker build --tag "azp-agent:arm64v8-alpine" --file "./azp-agent-arm64v8-alpine.dockerfile" .

Run the docker agent (don't forget to pre-define organization, pool, and token variables) —

docker run -e AZP_URL="https://dev.azure.com/${organization}" -e AZP_POOL="${pool}" -e AZP_AGENT_NAME="Docker Agent - arm64v8 Alpine" -e AZP_TOKEN="${token}" --name "azp-agent-arm64v8-alpine" azp-agent:arm64v8-alpine