freshclam may interfere with gpg --verify
gus-asf opened this issue · comments
Describe the bug
While preparing the Apache Solr 9.6 release I ran into an issue. Attempting to build a docker image based on the RC with the below command (which will cease to work once the RC is completed), The image would not build and complained that it could not connect to the gpg agent.
Killing freshclam resolved the problem
How to reproduce the problem
ps shows freshclam running
gus@ns-l1:~$ ps aux | grep clamav
clamav 1806 0.0 0.0 59124 14080 ? Ss 09:01 0:00 /usr/bin/freshclam -d --foreground=true
Attempt to build docker image for Solr
SOLR_DOWNLOAD_SERVER=https://dist.apache.org/repos/dist/dev/solr/solr-9.6.0-RC1-rev-f8e5a93c11267e13b7b43005a428bfb910ac6e57/solr && \
docker build $SOLR_DOWNLOAD_SERVER/9.6.0/docker/Dockerfile.official-full \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.6.0-1 && \
docker build $SOLR_DOWNLOAD_SERVER/9.6.0/docker/Dockerfile.official-slim \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.6.0-1-slim
This failed with
13.95 gpg: /tmp/gnupg_home/trustdb.gpg: trustdb created
13.95 gpg: key 140BC45803B03F7F: public key "Patrick Gustav Heck (CODE SIGNING KEY) <gus@apache.org>" imported
19.95 gpg: can't connect to the agent: IPC connect call failed
19.95 gpg: Total number processed: 1
19.95 gpg: imported: 1
That appears to correspond to this line:
67 | >>> gpg --batch --verify "/opt/$SOLR_BINARY.asc" "/opt/$SOLR_BINARY"; \
However for completeness the full output was:
------
context:30
--------------------
29 |
30 | >>> RUN set -ex; \
31 | >>> apt-get update; \
32 | >>> apt-get -y --no-install-recommends install wget gpg gnupg dirmngr; \
33 | >>> rm -rf /var/lib/apt/lists/*; \
34 | >>> export SOLR_BINARY="solr-$SOLR_VERSION$SOLR_DIST.tgz"; \
35 | >>> MAX_REDIRECTS=3; \
36 | >>> case "${SOLR_DOWNLOAD_SERVER}" in \
37 | >>> (*"apache.org"*);; \
38 | >>> (*) \
39 | >>> # If a non-ASF URL is provided, allow more redirects and skip GPG step.
40 | >>> MAX_REDIRECTS=4 && \
41 | >>> SKIP_GPG_CHECK=true;; \
42 | >>> esac; \
43 | >>> export DOWNLOAD_URL="$SOLR_DOWNLOAD_SERVER/$SOLR_VERSION/$SOLR_BINARY"; \
44 | >>> echo "downloading $DOWNLOAD_URL"; \
45 | >>> if ! wget -t 10 --max-redirect $MAX_REDIRECTS --retry-connrefused -nv "$DOWNLOAD_URL" -O "/opt/$SOLR_BINARY"; then rm -f "/opt/$SOLR_BINARY"; fi; \
46 | >>> if [ ! -f "/opt/$SOLR_BINARY" ]; then echo "failed download attempt for $SOLR_BINARY"; exit 1; fi; \
47 | >>> echo "$SOLR_SHA512 */opt/$SOLR_BINARY" | sha512sum -c -; \
48 | >>> if [ -z "$SKIP_GPG_CHECK" ]; then \
49 | >>> # Setup GPG \
50 | >>> export GNUPGHOME="/tmp/gnupg_home"; \
51 | >>> mkdir -p "$GNUPGHOME"; \
52 | >>> chmod 700 "$GNUPGHOME"; \
53 | >>> echo "disable-ipv6" >> "$GNUPGHOME/dirmngr.conf"; \
54 | >>> if [ -n "$SOLR_KEYS" ]; then \
55 | >>> # Install all Solr GPG Keys to start
56 | >>> wget -nv "https://downloads.apache.org/solr/KEYS" -O- | \
57 | >>> gpg --batch --import --key-origin 'url,https://downloads.apache.org/solr/KEYS'; \
58 | >>> # Save just the release key
59 | >>> release_keys="$(gpg --batch --export -a ${SOLR_KEYS})"; \
60 | >>> rm -rf "$GNUPGHOME"/*; \
61 | >>> echo "${release_keys}" | gpg --batch --import; \
62 | >>> fi; \
63 | >>> # Do GPG Checks
64 | >>> echo "downloading $DOWNLOAD_URL.asc"; \
65 | >>> wget -nv "$DOWNLOAD_URL.asc" -O "/opt/$SOLR_BINARY.asc"; \
66 | >>> (>&2 ls -l "/opt/$SOLR_BINARY" "/opt/$SOLR_BINARY.asc"); \
67 | >>> gpg --batch --verify "/opt/$SOLR_BINARY.asc" "/opt/$SOLR_BINARY"; \
68 | >>> # Cleanup GPG
69 | >>> { command -v gpgconf; gpgconf --kill all || :; }; \
70 | >>> rm -r "$GNUPGHOME"; \
71 | >>> else \
72 | >>> echo "Skipping GPG validation due to non-Apache build"; \
73 | >>> fi; \
74 | >>> tar -C /opt --extract --preserve-permissions --file "/opt/$SOLR_BINARY"; \
75 | >>> rm "/opt/$SOLR_BINARY"*; \
76 | >>> apt-get -y remove gpg dirmngr && apt-get -y autoremove;
77 |
--------------------
ERROR: failed to solve: process "/bin/sh -c set -ex; apt-get update; apt-get -y --no-install-recommends install wget gpg gnupg dirmngr; rm -rf /var/lib/apt/lists/*; export SOLR_BINARY=\"solr-$SOLR_VERSION$SOLR_DIST.tgz\"; MAX_REDIRECTS=3; case \"${SOLR_DOWNLOAD_SERVER}\" in (*\"apache.org\"*);; (*) MAX_REDIRECTS=4 && SKIP_GPG_CHECK=true;; esac; export DOWNLOAD_URL=\"$SOLR_DOWNLOAD_SERVER/$SOLR_VERSION/$SOLR_BINARY\"; echo \"downloading $DOWNLOAD_URL\"; if ! wget -t 10 --max-redirect $MAX_REDIRECTS --retry-connrefused -nv \"$DOWNLOAD_URL\" -O \"/opt/$SOLR_BINARY\"; then rm -f \"/opt/$SOLR_BINARY\"; fi; if [ ! -f \"/opt/$SOLR_BINARY\" ]; then echo \"failed download attempt for $SOLR_BINARY\"; exit 1; fi; echo \"$SOLR_SHA512 */opt/$SOLR_BINARY\" | sha512sum -c -; if [ -z \"$SKIP_GPG_CHECK\" ]; then export GNUPGHOME=\"/tmp/gnupg_home\"; mkdir -p \"$GNUPGHOME\"; chmod 700 \"$GNUPGHOME\"; echo \"disable-ipv6\" >> \"$GNUPGHOME/dirmngr.conf\"; if [ -n \"$SOLR_KEYS\" ]; then wget -nv \"https://downloads.apache.org/solr/KEYS\" -O- | gpg --batch --import --key-origin 'url,https://downloads.apache.org/solr/KEYS'; release_keys=\"$(gpg --batch --export -a ${SOLR_KEYS})\"; rm -rf \"$GNUPGHOME\"/*; echo \"${release_keys}\" | gpg --batch --import; fi; echo \"downloading $DOWNLOAD_URL.asc\"; wget -nv \"$DOWNLOAD_URL.asc\" -O \"/opt/$SOLR_BINARY.asc\"; (>&2 ls -l \"/opt/$SOLR_BINARY\" \"/opt/$SOLR_BINARY.asc\"); gpg --batch --verify \"/opt/$SOLR_BINARY.asc\" \"/opt/$SOLR_BINARY\"; { command -v gpgconf; gpgconf --kill all || :; }; rm -r \"$GNUPGHOME\"; else echo \"Skipping GPG validation due to non-Apache build\"; fi; tar -C /opt --extract --preserve-permissions --file \"/opt/$SOLR_BINARY\"; rm \"/opt/$SOLR_BINARY\"*; apt-get -y remove gpg dirmngr && apt-get -y autoremove;" did not complete successfully: exit code: 2
After killing clamav as shown below the docker build succeeded:
gus@ns-l1:~$ ps aux | grep clamav
clamav 1806 0.0 0.0 59124 14080 ? Ss 09:01 0:00 /usr/bin/freshclam -d --foreground=true
gus 11896 0.0 0.0 9212 2304 pts/0 S+ 09:09 0:00 grep --color=auto clamav
gus@ns-l1:~$ kill 1806
bash: kill: (1806) - Operation not permitted
gus@ns-l1:~$ sudo kill -9 1806
[sudo] password for gus:
gus@ns-l1:~$ ps aux | grep clamav
gus 11940 0.0 0.0 9080 2304 pts/0 S+ 09:09 0:00 grep --color=auto clamav
gus@ns-l1:~$ SOLR_DOWNLOAD_SERVER=https://dist.apache.org/repos/dist/dev/solr/solr-9.6.0-RC1-rev-f8e5a93c11267e13b7b43005a428bfb910ac6e57/solr && docker build $SOLR_DOWNLOAD_SERVER/9.6.0/docker/Dockerfile.official-full --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER -t solr-rc:9.6.0-1 && docker build $SOLR_DOWNLOAD_SERVER/9.6.0/docker/Dockerfile.official-slim --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER -t solr-rc:9.6.0-1-slim
[+] Building 29.1s (10/10) FINISHED docker:default
=> CACHED [internal] load remote build context 0.2s
=> [internal] load metadata for docker.io/library/eclipse-temurin:17-jre-jammy 0.3s
=> CACHED [1/7] FROM docker.io/library/eclipse-temurin:17-jre-jammy@sha256:1b646daef966395c93995e73347d4c7c726c9ddba8695e984cd8dcf5d8b5b253 0.0s
=> [2/7] RUN set -ex; apt-get update; apt-get -y --no-install-recommends install wget gpg gnupg dirmngr; rm -rf /var/lib/apt/lists/*; export SOLR_BINARY="solr-9.6.0.tgz"; MAX_RED 20.7s
=> [3/7] RUN set -ex; groupadd -r --gid "8983" "solr"; useradd -r --uid "8983" --gid "8983" "solr" 0.4s
=> [4/7] RUN set -ex; (cd /opt; ln -s solr-*/ solr); rm -Rf /opt/solr/docs /opt/solr/docker/Dockerfile; 0.4s
=> [5/7] RUN set -ex; mkdir -p /opt/solr/server/solr/lib /docker-entrypoint-initdb.d; cp /opt/solr/bin/solr.in.sh /etc/default/solr.in.sh; mv /opt/solr/bin/solr.in.sh /opt/solr/bin/so 0.5s
=> [6/7] RUN set -ex; apt-get update; apt-get -y --no-install-recommends install acl lsof procps wget netcat gosu tini jattach; rm -rf /var/lib/apt/lists/*; 5.3s
=> [7/7] WORKDIR /opt/solr 0.1s
=> exporting to image 0.9s
=> => exporting layers 0.9s
=> => writing image sha256:e426f6991bd325a5b01a31e90f3304ce45a787004c34fc6a90a2aa86c5193afc 0.0s
=> => naming to docker.io/library/solr-rc:9.6.0-1 0.0s
[+] Building 19.3s (10/10) FINISHED docker:default
=> [internal] load remote build context 0.2s
=> [internal] load metadata for docker.io/library/eclipse-temurin:17-jre-jammy 0.2s
=> CACHED [1/7] FROM docker.io/library/eclipse-temurin:17-jre-jammy@sha256:1b646daef966395c93995e73347d4c7c726c9ddba8695e984cd8dcf5d8b5b253 0.0s
=> [2/7] RUN set -ex; apt-get update; apt-get -y --no-install-recommends install wget gpg gnupg dirmngr; rm -rf /var/lib/apt/lists/*; export SOLR_BINARY="solr-9.6.0-slim.tgz"; MA 11.3s
=> [3/7] RUN set -ex; groupadd -r --gid "8983" "solr"; useradd -r --uid "8983" --gid "8983" "solr" 0.5s
=> [4/7] RUN set -ex; (cd /opt; ln -s solr-*/ solr); rm -Rf /opt/solr/docs /opt/solr/docker/Dockerfile; 0.5s
=> [5/7] RUN set -ex; mkdir -p /opt/solr/server/solr/lib /docker-entrypoint-initdb.d; cp /opt/solr/bin/solr.in.sh /etc/default/solr.in.sh; mv /opt/solr/bin/solr.in.sh /opt/solr/bin/so 0.5s
=> [6/7] RUN set -ex; apt-get update; apt-get -y --no-install-recommends install acl lsof procps wget netcat gosu tini jattach; rm -rf /var/lib/apt/lists/*; 5.6s
=> [7/7] WORKDIR /opt/solr 0.1s
=> exporting to image 0.5s
=> => exporting layers 0.4s
=> => writing image sha256:4933f3684276f1f7e239046e4978ab223df902f428b7af3a20300fca264a94c3 0.0s
=> => naming to docker.io/library/solr-rc:9.6.0-1-slim
Replace this text with the output from the ClamAV command:
clamconf -n
gus@ns-l1:~$ clamconf -n
Command 'clamconf' not found, but can be installed with:
sudo apt install clamav-daemon
However
gus@ns-l1:~$ clamscan --version
ClamAV 0.103.11/27254/Tue Apr 23 04:23:39 2024
gus@ns-l1:~$ freshclam --version
ClamAV 0.103.11/27254/Tue Apr 23 04:23:39 2024
gus@ns-l1:~$ gpg --version
gpg (GnuPG) 2.2.27
libgcrypt 1.9.4
Copyright (C) 2021 Free Software Foundation, Inc.
License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Home: /home/gus/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2
gus@ns-l1:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS"
I got lucky and found this quite quickly due to a stack overflow that mentioned windows defender and so I thought to try killing clamav, but it sure could have been a stumper
freshclam is not a scanning service. ClamOnAcc is the on-access (real-time) scanning service) and relies on clamd (clamav-daemon) which you do not have installed.
freshclam is a service to occasionally download updates to the clamav signatures. The freshclam service should have no affect on using gpg. This seems like a coincidence to me.
Hmm seems you are right the error seems to crop up ~50-60% of the time and of course initially, I got (un)lucky and failed like 4 times in a row before killing the freshclam process and it suddenly working, sorry for the noise.