Update OS used to build .NET package

Question

Update OS used to build .NET package

bfren opened this issue 2 years ago · comments

Currently the build and test jobs for the .NET package run on Ubuntu 16.04 - now Xenial is no longer receiving maintenance updates this should change.

bfren · Answer 1 · Fri Nov 25 2022 18:13:49 GMT+0800 (China Standard Time)

The issue is the run-test-binaries-os-versions job, which tests compatibility with CentOS 7 and Debian 10.

So far I have used the following for the build-linux-glibc-xx jobs:

Ubuntu 20.04 fails
Ubuntu 22.04 fails
Debian 10 fails

Each time the error is:

Run chmod +x .libsodium-builds/linux-x64/Tests
  chmod +x .libsodium-builds/linux-x64/Tests
  .libsodium-builds/linux-x64/Tests
  shell: sh -e {0}
  env:
    DOTNET_SYSTEM_GLOBALIZATION_INVARIANT: 1
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'libsodium' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibsodium: cannot open shared object file: No such file or directory
   at Program.sodium_version_string()
   at Program.Main() in /__w/libsodium/libsodium/.libsodium-test/Program.cs:line 8
/__w/_temp/10cd1428-5f50-4743-9b21-01f1bbd2452a.sh: line 2:    39 Aborted                 (core dumped) .libsodium-builds/linux-x64/Tests
Error: Process completed with exit code 134.

However using Debian 10 as the build platform (supported until 30 June 2024) means the Debian 10 tests pass (all the other tests pass on all the above) - so the main problem is CentOS 7.

If CentOS 7 compatibility is important, is it worth considering a build process for CentOS 7 separate to other platforms?

Frank Denis · Answer 2 · Fri Nov 25 2022 18:38:08 GMT+0800 (China Standard Time)

I'd love to ditch CentOS 7 compatibility, but people are going to complain that it is still supported by the vendor.

ektrah · Answer 3 · Fri Nov 25 2022 19:42:06 GMT+0800 (China Standard Time)

.NET unfortunately masks the actual error: It tries to load libsodium from a number of places and using different spellings (e.g., by adding a .so suffix and/or a lib prefix to the library name). If it cannot load and initialize the library, it throws a DllNotFoundException saying "liblibsodium: cannot open shared object file: No such file or directory", even if the library exists in one of the places and spellings.

The solution to finding the actual error is given in the exception message: "In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable". My guess is that this will show an "Undefined reference to memcpy@GLIBC_2.14" error.

ektrah · Answer 4 · Sun Nov 27 2022 21:15:21 GMT+0800 (China Standard Time)

When compiling libsodium.so on ubuntu:22.04, loading that file fails on both centos:7 and debian:10 with the following error:

checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.33' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
/lib64/libc.so.6: error: version lookup error: version `GLIBC_2.33' not found (required by /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so) (fatal)

This is independent of whether __asm__(".symver memcpy,memcpy@GLIBC_2.2.5"); is present or not.

When compiling on ubuntu:20.04 or ubuntu:18.04 and loading on centos:7:

checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.2.5' in file /lib64/libpthread.so.0 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.14' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.3.4' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.4' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.25' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
/lib64/libc.so.6: error: version lookup error: version `GLIBC_2.25' not found (required by /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so) (fatal)

When compiling on ubuntu:16.04, everything works (without __asm__(".symver memcpy,memcpy@GLIBC_2.2.5");).

So it seems that, until CentOS 7 reaches end-of-life (or .NET drops support for it), we're stuck with compiling libsodium.so for .NET on ubuntu:16.04...

glibc versions of distributions:

Distribution	glibc version	EOL
CentOS 7.7.1908	2.17	2024-06
Ubuntu 16.04	2.23	2026-04
Ubuntu 18.04	2.27	2028-04
Debian 10	2.28	2022-09
Ubuntu 20.04	2.31	2030-04
Debian 11	2.31	2024-07
Fedora 35	2.34	2022-12
Fedora 36	2.35	2023-05
Ubuntu 22.04	2.35	2032-04

libc compatibility of .NET itself:

x64: glibc 2.17 (from CentOS 7)

Arm32, Arm64: glibc 2.23 (from Ubuntu 16.04)

Alpine (x64 and Arm64): musl 1.2.2 (from Alpine 3.13)

Frank Denis · Answer 5 · Mon Nov 28 2022 03:17:53 GMT+0800 (China Standard Time)

Another option is to compile with Zig, that can cross-compile to many targets, including specific libc versions:

Example:

zig build -Drelease-fast -Dtarget=x86_64-linux-gnu.2.17

Frank Denis · Answer 6 · Mon Nov 28 2022 07:13:23 GMT+0800 (China Standard Time)

Looks like this is the best way to go.

dotnet-core.yml is now using zig to build Linux binaries.

Compilation can thus be done on current Linux distributions, while targeting older glibc versions. So, we target glibc 2.17 for x86_64 and glibc 2.23 for ARM, according to your table.

Compilation to aarch64 and arm with musl was fixed by the way (previously, the aarch64 build was actually an x86_64 one).

Frank Denis · Answer 7 · Mon Nov 28 2022 07:23:03 GMT+0800 (China Standard Time)

Adding Windows ARM would probably also be nice.

zig build -Drelease-fast -Dtarget=aarch64-windows gives us a DLL and a .LIB for Windows ARM. But I guess this is impossible to test on GitHub CI, right?

bfren · Answer 8 · Fri Dec 09 2022 18:41:49 GMT+0800 (China Standard Time)

Is 1.0.18.3 waiting on this?

Frank Denis · Answer 9 · Sat Dec 10 2022 07:04:58 GMT+0800 (China Standard Time)

1.0.18.3 has been published :)

ektrah · Answer 10 · Sun Dec 11 2022 20:58:49 GMT+0800 (China Standard Time)

Some unit tests in NSec that passed with 1.0.18.2 are now failing with 1.0.18.3 ☹️

It looks like something changed in or around crypto_aead_aes256gcm_decrypt?

ektrah · Answer 11 · Sun Dec 11 2022 21:37:58 GMT+0800 (China Standard Time)

And it seems the pre-compiled binary built for linux-musl-arm isn't actually included in the NuGet package...

Frank Denis · Answer 12 · Sun Dec 11 2022 21:49:14 GMT+0800 (China Standard Time)

Ah, looks like that has to be added to libsodium.pkgproj.

bfren · Answer 13 · Sun Dec 11 2022 23:05:36 GMT+0800 (China Standard Time)

Ahh, that must also have got clobbered like the version bump I did.

bfren · Answer 14 · Sun Dec 11 2022 23:24:18 GMT+0800 (China Standard Time)

Some unit tests in NSec that passed with 1.0.18.2 are now failing with 1.0.18.3 ☹️

It looks like something changed in or around crypto_aead_aes256gcm_decrypt?

Is this to do with 408125a?

Frank Denis · Answer 15 · Mon Dec 12 2022 03:23:09 GMT+0800 (China Standard Time)

Some unit tests in NSec that passed with 1.0.18.2 are now failing with 1.0.18.3 ☹️
It looks like something changed in or around crypto_aead_aes256gcm_decrypt?

Is this to do with 408125a?

Very unlikely. This code doesn't exist in stable.

Frank Denis · Answer 16 · Mon Dec 12 2022 03:26:28 GMT+0800 (China Standard Time)

On a failed decryption, the message buffer is now filled with 0xd0 instead of 0x00 (in the tradition of OpenBSD, making it more obvious that something went wrong instead of an all-null content).

The NSec test should check that the output buffer doesn't match the plaintext rather than expect a specific output on failure.

Frank Denis · Answer 17 · Mon Dec 12 2022 03:47:01 GMT+0800 (China Standard Time)

What do the other failing tests do?

ektrah · Answer 18 · Mon Dec 12 2022 04:31:20 GMT+0800 (China Standard Time)

All three of the other failing tests are decrypting a valid ciphertext using crypto_aead_aes256gcm_decrypt. However:

In DecryptWithAdOverlapping, m (length mlen) and ad (length adlen) overlap in memory (m < ad < ad+adlen < m+mlen).
In DecryptWithSpanInPlace, m and c point at the same memory location (m == c).
In DecryptWithSpanOutOfPlace, none of the buffers overlap.

Previously, in each case, the function returned zero; now it returns a non-zero value.

The last case puzzles me the most because it is the simplest possible test case: encrypt a plaintext and decrypt the resulting ciphertext with exactly the same key, nonce and additional data. How could that possibly fail? 😕

ektrah · Answer 19 · Mon Dec 12 2022 05:21:14 GMT+0800 (China Standard Time)

The simplest possible test case fails if clen >= 256.

Frank Denis · Answer 20 · Mon Dec 12 2022 06:50:53 GMT+0800 (China Standard Time)

Oops! That should be fixed by a5ea347

Our test suite was missing tests with long inputs. This case is now tested.