Chacha/box tests segfault when compiling with LLVM Polly

[Mitchell Skaggs]

Clang 16.0.1 flags -march=skylake-avx512 -mtune=skylake-avx512 -O3 -mllvm -polly, cross-compiling libsodium v1.0.18 to i686 via Nix.

It's entirely possible this is a LLVM/Polly bug, but I thought I'd make a note of it here in case someone else searched for this highly-specific issue.

Test logs:

===================================================
   libsodium 1.0.18: test/default/test-suite.log
===================================================

# TOTAL: 77
# PASS:  64
# SKIP:  0
# XFAIL: 0
# FAIL:  13
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: aead_chacha20poly1305
===========================

FAIL aead_chacha20poly1305 (exit status: 139)

FAIL: aead_chacha20poly13052
============================

FAIL aead_chacha20poly13052 (exit status: 139)

FAIL: aead_xchacha20poly1305
============================

FAIL aead_xchacha20poly1305 (exit status: 139)

FAIL: box7
==========

FAIL box7 (exit status: 139)

FAIL: box8
==========

FAIL box8 (exit status: 139)

FAIL: box_easy2
===============

FAIL box_easy2 (exit status: 139)

FAIL: box_seal
==============

FAIL box_seal (exit status: 139)

FAIL: chacha20
==============

FAIL chacha20 (exit status: 139)

FAIL: secretbox7
================

FAIL secretbox7 (exit status: 139)

FAIL: secretbox8
================

FAIL secretbox8 (exit status: 139)

FAIL: secretbox_easy2
=====================

FAIL secretbox_easy2 (exit status: 139)

FAIL: secretstream
==================

FAIL secretstream (exit status: 139)

FAIL: xchacha20
===============

FAIL xchacha20 (exit status: 139)

[Frank Denis]

I don't think Polly is expected to play well with code using architecture-specific vector instructions.

libsodium code (and cryptographic code in general) is full of already vectorized implementations, unrolled loops, and the rest can hardly be vectorized. That's the worst possible case for Polly, so performance regressions are more likely than performance gains.

Most importantly, Polly is mainly a research project. And regardless of the test suite results, using this in a crypto library (or any experimental compiler optimization pass) is a terrible idea. An edge case not caught by the test suite could have catastrophic implications.

Libsodium may work with Polly if you disable all the AVX/AVX2/AVX512 implementations. But then, you'll end up with code that's very slow.

If you want to investigate that issue, go ahead. But unless it looks like a ton of work for something that's not worth it and that no one besides you is, or should be doing.

You are a smart person and know that nothing is going to happen by keeping that issue open, unless you get your hands dirty, or find a way to reproduce the same test failures with common, non-experimental optimization passes.

In the meantime, the documentation will be updated to explicitly discourage usage of Polly and other experimental optimization passes.