[OT] pynacl build failure
NathanDotTo opened this issue · comments
I have a test failure in a build for a Docker container on macOS 12.4. The instructions are to post the error here, so I am.
I have included below the following:
- Error log snippet.
- Dockerfile
- Docker build command
- Requirements.txt
So you should be able to replicate this.
The error log is:
280.1 CC shorthash.o
#13 280.1 CCLD shorthash
#13 280.1 CC sign.o
#13 280.1 CCLD sign
#13 280.1 CC sodium_core.o
#13 280.1 CCLD sodium_core
#13 280.1 CC sodium_utils.o
#13 280.1 CCLD sodium_utils
#13 280.1 CC sodium_version.o
#13 280.1 CCLD sodium_version
#13 280.1 CC stream.o
#13 280.1 CCLD stream
#13 280.1 CC stream2.o
#13 280.1 CCLD stream2
#13 280.1 CC stream3.o
#13 280.1 CCLD stream3
#13 280.1 CC stream4.o
#13 280.1 CCLD stream4
#13 280.1 CC verify1.o
#13 280.1 CCLD verify1
#13 280.1 CC sodium_utils2.o
#13 280.1 CCLD sodium_utils2
#13 280.1 CC sodium_utils3.o
#13 280.1 CCLD sodium_utils3
#13 280.1 CC core_ed25519.o
#13 280.1 CCLD core_ed25519
#13 280.1 CC pwhash_scrypt.o
#13 280.1 In file included from /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/cmptest.h:9,
#13 280.1 from /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/pwhash_scrypt.c:3:
#13 280.1 In function 'str_tests',
#13 280.1 inlined from 'xmain' at /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/pwhash_scrypt.c:376:5:
#13 280.1 /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/pwhash_scrypt.c:362:12: warning: 'crypto_pwhash_scryptsalsa208sha256_str_needs_rehash' reading 102 bytes from a region of size 1 [-Wstringop-overread]
#13 280.1 362 | assert(crypto_pwhash_scryptsalsa208sha256_str_needs_rehash
#13 280.1 | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#13 280.1 363 | ("", OPSLIMIT, MEMLIMIT) == -1);
#13 280.1 | ~~~~~~~~~~~~~~~~~~~~~~~~
#13 280.1 /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/pwhash_scrypt.c: In function 'xmain':
#13 280.1 /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/pwhash_scrypt.c:362:12: note: referencing argument 1 of type 'const char *'
#13 280.1 In file included from /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/src/libsodium/include/sodium.h:64,
#13 280.1 from /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/cmptest.h:16,
#13 280.1 from /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/test/default/pwhash_scrypt.c:3:
#13 280.1 /tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/src/libsodium/src/libsodium/include/sodium/crypto_pwhash_scryptsalsa208sha256.h:111:5: note: in a call to function 'crypto_pwhash_scryptsalsa208sha256_str_needs_rehash'
#13 280.1 111 | int crypto_pwhash_scryptsalsa208sha256_str_needs_rehash(const char str[crypto_pwhash_scryptsalsa208sha256_STRBYTES],
#13 280.1 | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#13 280.1 CCLD pwhash_scrypt
#13 280.1 CC pwhash_scrypt_ll.o
#13 280.1 CCLD pwhash_scrypt_ll
#13 280.1 CC scalarmult_ed25519.o
#13 280.1 CCLD scalarmult_ed25519
#13 280.1 CC siphashx24.o
#13 280.1 CCLD siphashx24
#13 280.1 CC xchacha20.o
#13 280.1 CCLD xchacha20
#13 280.1 make[3]: Leaving directory '/tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/build/temp.linux-aarch64-cpython-310/test/default'
#13 280.1 make check-TESTS
#13 280.1 make[3]: Entering directory '/tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/build/temp.linux-aarch64-cpython-310/test/default'
#13 280.1 make[4]: Entering directory '/tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/build/temp.linux-aarch64-cpython-310/test/default'
#13 280.1 PASS: aead_aes256gcm
#13 280.1 PASS: aead_chacha20poly1305
#13 280.1 PASS: aead_xchacha20poly1305
#13 280.1 PASS: auth
#13 280.1 PASS: auth2
#13 280.1 PASS: auth3
#13 280.1 PASS: auth5
#13 280.1 PASS: auth6
#13 280.1 PASS: auth7
#13 280.1 PASS: box
#13 280.1 PASS: box2
#13 280.1 PASS: box7
#13 280.1 PASS: box8
#13 280.1 PASS: box_easy
#13 280.1 PASS: box_easy2
#13 280.1 PASS: box_seal
#13 280.1 PASS: box_seed
#13 280.1 PASS: chacha20
#13 280.1 PASS: codecs
#13 280.1 PASS: core1
#13 280.1 PASS: core2
#13 280.1 PASS: core3
#13 280.1 PASS: core4
#13 280.1 PASS: core5
#13 280.1 PASS: core6
#13 280.1 PASS: ed25519_convert
#13 280.1 PASS: generichash
#13 280.1 PASS: generichash2
#13 280.1 PASS: generichash3
#13 280.1 PASS: hash
#13 280.1 PASS: hash3
#13 280.1 PASS: kdf
#13 280.1 PASS: keygen
#13 280.1 PASS: kx
#13 280.1 PASS: metamorphic
#13 280.1 PASS: misuse
#13 280.1 PASS: onetimeauth
#13 280.1 PASS: onetimeauth2
#13 280.1 PASS: onetimeauth7
#13 280.1 PASS: pwhash_argon2i
#13 280.1 PASS: pwhash_argon2id
#13 280.1 PASS: randombytes
#13 280.1 PASS: scalarmult
#13 280.1 PASS: scalarmult2
#13 280.1 PASS: scalarmult5
#13 280.1 PASS: scalarmult6
#13 280.1 PASS: scalarmult7
#13 280.1 PASS: secretbox
#13 280.1 PASS: secretbox2
#13 280.1 PASS: secretbox7
#13 280.1 PASS: secretbox8
#13 280.1 PASS: secretbox_easy
#13 280.1 PASS: secretbox_easy2
#13 280.1 PASS: secretstream
#13 280.1 PASS: shorthash
#13 280.1 PASS: sign
#13 280.1 PASS: sodium_core
#13 280.1 PASS: sodium_utils
#13 280.1 PASS: sodium_version
#13 280.1 PASS: stream
#13 280.1 PASS: stream2
#13 280.1 PASS: stream3
#13 280.1 PASS: stream4
#13 280.1 PASS: verify1
#13 280.1 PASS: sodium_utils2
#13 280.1 PASS: sodium_utils3
#13 280.1 PASS: core_ed25519
#13 280.1 FAIL: pwhash_scrypt
#13 280.1 PASS: pwhash_scrypt_ll
#13 280.1 PASS: scalarmult_ed25519
#13 280.1 PASS: siphashx24
#13 280.1 PASS: xchacha20
#13 280.1 ============================================================================
#13 280.1 Testsuite summary for libsodium 1.0.16
#13 280.1 ============================================================================
#13 280.1 # TOTAL: 72
#13 280.1 # PASS: 71
#13 280.1 # SKIP: 0
#13 280.1 # XFAIL: 0
#13 280.1 # FAIL: 1
#13 280.1 # XPASS: 0
#13 280.1 # ERROR: 0
#13 280.1 ============================================================================
#13 280.1 See test/default/test-suite.log
#13 280.1 Please report to https://github.com/jedisct1/libsodium/issues
#13 280.1 ============================================================================
#13 280.1 make[4]: *** [Makefile:1805: test-suite.log] Error 1
#13 280.1 make[4]: Leaving directory '/tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/build/temp.linux-aarch64-cpython-310/test/default'
#13 280.1 make[3]: *** [Makefile:1913: check-TESTS] Error 2
#13 280.1 make[3]: Leaving directory '/tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/build/temp.linux-aarch64-cpython-310/test/default'
#13 280.1 make[2]: *** [Makefile:2481: check-am] Error 2
#13 280.1 make[2]: Leaving directory '/tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/build/temp.linux-aarch64-cpython-310/test/default'
#13 280.1 make[1]: *** [Makefile:402: check-recursive] Error 1
#13 280.1 make[1]: Leaving directory '/tmp/pip-install-liiza1vw/pynacl_65a98221691b42a498a1e4d13e0ef090/build/temp.linux-aarch64-cpython-310/test'
#13 280.1 make: *** [Makefile:515: check-recursive] Error 1
#13 280.1 Traceback (most recent call last):
#13 280.1 File "/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
#13 280.1 main()
#13 280.1 File "/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
#13 280.1 json_out['return_val'] = hook(**hook_input['kwargs'])
#13 280.1 File "/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
#13 280.1 return _build_backend().build_wheel(wheel_directory, config_settings,
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 244, in build_wheel
#13 280.1 return self._build_with_temp_dir(['bdist_wheel'], '.whl',
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 229, in _build_with_temp_dir
#13 280.1 self.run_setup()
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 281, in run_setup
#13 280.1 super(_BuildMetaLegacyBackend,
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 174, in run_setup
#13 280.1 exec(compile(code, __file__, 'exec'), locals())
#13 280.1 File "setup.py", line 208, in <module>
#13 280.1 setup(
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
#13 280.1 return distutils.core.setup(**attrs)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 177, in setup
#13 280.1 return run_commands(dist)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
#13 280.1 dist.run_commands()
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
#13 280.1 self.run_command(cmd)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1229, in run_command
#13 280.1 super().run_command(command)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
#13 280.1 cmd_obj.run()
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 299, in run
#13 280.1 self.run_command('build')
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
#13 280.1 self.distribution.run_command(command)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1229, in run_command
#13 280.1 super().run_command(command)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
#13 280.1 cmd_obj.run()
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/command/build.py", line 24, in run
#13 280.1 super().run()
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
#13 280.1 self.run_command(cmd_name)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
#13 280.1 self.distribution.run_command(command)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1229, in run_command
#13 280.1 super().run_command(command)
#13 280.1 File "/tmp/pip-build-env-7n67_1vy/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
#13 280.1 cmd_obj.run()
#13 280.1 File "setup.py", line 179, in run
#13 280.1 subprocess.check_call(["make", "check"] + make_args, cwd=build_temp)
#13 280.1 File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
#13 280.1 raise CalledProcessError(retcode, cmd)
#13 280.1 subprocess.CalledProcessError: Command '['make', 'check']' returned non-zero exit status 2.
#13 280.1 [end of output]
#13 280.1
#13 280.1 note: This error originates from a subprocess, and is likely not a problem with pip.
#13 280.1 ERROR: Failed building wheel for PyNaCl
#13 280.1 Building wheel for PyYAML (setup.py): started
#13 281.7 Building wheel for PyYAML (setup.py): finished with status 'done'
#13 281.7 Created wheel for PyYAML: filename=PyYAML-5.1.2-cp310-cp310-linux_aarch64.whl size=44118 sha256=654719b8ea274535235831f30597cfb8438285fbd9f6a028e16e583ba58d354b
#13 281.7 Stored in directory: /root/.cache/pip/wheels/1c/77/3e/dcf9cc5e235189dedcf5f1736a14caaac20267a5bb846c8ce1
#13 281.7 Building wheel for SecretStorage (setup.py): started
#13 283.1 Building wheel for SecretStorage (setup.py): finished with status 'done'
#13 283.1 Created wheel for SecretStorage: filename=SecretStorage-2.3.1-py3-none-any.whl size=13972 sha256=ec5dc0dd730a41ce91747a8d6bc311c89d3d9665cd60b606818323d68d431e67
#13 283.1 Stored in directory: /root/.cache/pip/wheels/e3/30/39/48f1f907ff866f5e0d010040eab0661bc74d304cb5fbfa624b
#13 283.1 Building wheel for ansible (setup.py): started
#13 321.8 Building wheel for ansible (setup.py): finished with status 'done'
#13 321.8 Created wheel for ansible: filename=ansible-5.9.0-py3-none-any.whl size=64500299 sha256=a06535dd4ccbf025c0709d61168597e1dde16f1d5b379f76d3a96b9edcb84e5d
#13 321.8 Stored in directory: /root/.cache/pip/wheels/00/16/f2/d73483648d7080202d841d0053e44897c52f2323766bc2b8dd
#13 322.1 Building wheel for lxml (setup.py): started
#13 402.2 Building wheel for lxml (setup.py): still running...
#13 447.1 Building wheel for lxml (setup.py): finished with status 'done'
#13 447.1 Created wheel for lxml: filename=lxml-4.9.0-cp310-cp310-linux_aarch64.whl size=7844934 sha256=79b3aca67d48a50ac6eab5985b0a26d9f6b30ab83299dafe39d9aa1b8b54b3a6
#13 447.1 Stored in directory: /root/.cache/pip/wheels/c5/d7/04/019492c5195adf8b92d0f6a598d42faf35438f678cee710431
#13 447.1 Building wheel for ansible-core (setup.py): started
#13 449.4 Building wheel for ansible-core (setup.py): finished with status 'done'
#13 449.4 Created wheel for ansible-core: filename=ansible_core-2.12.7-py3-none-any.whl size=2078314 sha256=19da5a081c248fcd20e65bf3bb261aa7327e2c4144f997bc10ddac33dec8a5a1
#13 449.4 Stored in directory: /root/.cache/pip/wheels/72/e7/a6/e71f04f473c2ca3b072e60099a211b1516a784a90a924e3191
#13 449.4 Building wheel for ruamel.yaml.clib (setup.py): started
#13 461.6 Building wheel for ruamel.yaml.clib (setup.py): finished with status 'done'
#13 461.6 Created wheel for ruamel.yaml.clib: filename=ruamel.yaml.clib-0.2.6-cp310-cp310-linux_aarch64.whl size=692022 sha256=e6ba825d47c8eab0214f43e62ace68b37ef7b784e886e1b9bebadbec2299a02f
#13 461.6 Stored in directory: /root/.cache/pip/wheels/cf/46/ea/4fa5d9baea561fb37620546385dfdaf5a8c37bb491dfb315cb
#13 461.6 Successfully built bcrypt ipaddress MarkupSafe ncclient pycparser pycrypto PyYAML SecretStorage ansible lxml ansible-core ruamel.yaml.clib
#13 461.6 Failed to build cffi PyNaCl
#13 461.6 ERROR: Could not build wheels for PyNaCl, which is required to install pyproject.toml-based projects
------
The Docker file is:
FROM alpine:3.16
ARG build_date=unspecified
ARG terraform_version=unspecified
ARG arch=unspecified
RUN apk add --no-cache gcc musl-dev make
RUN apk add --no-cache python3
RUN if [ ! -e /usr/bin/python ]; then ln -sf python3 /usr/bin/python ; fi
RUN python3 -m ensurepip
RUN pip3 install --no-cache --upgrade pip setuptools wheel
RUN apk --update add git sshpass libffi-dev libxml2-dev libxslt-dev python3-dev openssl-dev openssh-keygen
COPY requirements.txt /tmp/requirements.txt
RUN pip install -r /tmp/requirements.txt
RUN apk --update add wget unzip cdrkit curl
RUN wget --quiet https://releases.hashicorp.com/terraform/${terraform_version}/terraform_${terraform_version}_linux_${arch}.zip
RUN unzip terraform_${terraform_version}_linux_${arch}.zip
RUN mv terraform /usr/bin
RUN rm terraform_${terraform_version}_linux_${arch}.zip
ENV ANSIBLE_HOST_KEY_CHECKING=false
ENV ANSIBLE_RETRY_FILES_ENABLED=false
ENV ANSIBLE_SSH_PIPELINING=true
ENV ANSIBLE_LOCAL_TMP=/tmp
ENV ANSIBLE_REMOTE_TMP=/tmp
WORKDIR /ansible
The Docker build command is:
cp ../requirements.txt .; docker build -t x-devops --build-arg build_date='date -Iseconds' --build-arg terraform_version=1.1.9 --build-arg arch=arm64 .
The requirements.txt is:
ansible>=4.2.0
ansible-lint==5.4.0
asn1crypto==0.24.0
bcrypt==3.1.7
certifi==2019.6.16
cffi==1.12.3
chardet==3.0.4
cryptography>=3.2
enum34==1.1.6
idna==2.6
ipaddress==1.0.17
jinja2>=2.11.3
jmespath==0.9.5
keyring==10.6.0
keyrings.alt==3.0
lxml>=4.6.5
MarkupSafe==1.1.1
ncclient==0.6.6
netaddr==0.7.19
paramiko>=2.10.1
pycparser==2.19
pycrypto==2.6.1
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyxdg==0.26
PyYAML==5.1.2
requests==2.28.0
scp==0.13.2
SecretStorage==2.3.1
selectors2==2.0.1
six==1.11.0
urllib3>=1.26.5
The full build log is here: https://gist.github.com/NathanDotTo/0b91a91c0e18d6410906a39126d2e614
Updating this dependency has helped me move on, albeit to a different error:
PyNaCl==1.5.0
For the record, the requirements.txt that eventually worked for me is:
ansible>=4.2.0
ansible-lint==5.4.0
asn1crypto==1.5.1
bcrypt==3.2.2
certifi==2022.6.15
cffi==1.15.0
chardet==3.0.4
cryptography>=3.2
enum34==1.1.6
idna==2.6
ipaddress==1.0.17
jinja2>=2.11.3
jmespath==0.9.5
keyring==10.6.0
keyrings.alt==3.0
lxml>=4.6.5
MarkupSafe==1.1.1
ncclient==0.6.6
netaddr==0.7.19
paramiko>=2.10.1
pycparser==2.19
pycrypto==2.6.1
PyNaCl==1.5.0
pyOpenSSL==19.0.0
pyxdg==0.26
PyYAML==5.1.2
requests==2.28.0
scp==0.13.2
SecretStorage==2.3.1
selectors2==2.0.1
six==1.11.0
urllib3>=1.26.5
The version 1.0.16 of libsodium you're trying to install is very old. Maybe there's a more recent version of pynacl than the one you are trying to install, that bundles a more modern libsodium?
pynacl is a different project, maintained independently. I don't have much experience with it, so if you keep having issues with it, I'd recommend reporting them to their repository: https://github.com/pyca/pynacl
See new requirements.txt above.