s2n_aead_aes_test times out occasionally
maddeleine opened this issue · comments
Security issue notifications
If you discover a potential security issue in s2n we ask that you notify
AWS Security via our vulnerability reporting page. Please do not create a public github issue.
Problem:
We have noticed the s2n_aead_aes_test will occasionally take way longer than usual to run and then timeout. This appears to be flaky as a retry fixes the problem. Here's an example of the failing test.
Solution:
Figure out what is causing the flakiness. It's possible it has something to do with not having aes-optimized hardware available, but that's just a theory. Investigation is needed.
After investigating this issue a bit, it seems that this isn't a problem with the aead_aes test itself, but rather an issue with the kTLS CI job running all tests very slowly sometimes. The slow CI job causes the aead_aes test to timeout since it's the slowest unit test.
I compared a successful kTLS test run to a kTLS test run with the aead_aes test timing out, and many of the other tests were also slow:
| Test | Success Duration (s) | Timeout Duration (s) | % Increase |
|---|---|---|---|
| s2n_quic_transport_params_extension_test | 0.05 | 521.34 | 1042580% |
| s2n_ssl_prf_test | 0.04 | 83.91 | 209675% |
| s2n_tls13_secrets_rfc8448_test | 0.07 | 36.38 | 51871% |
| s2n_ktls_io_sendfile_test | 0.08 | 41.32 | 51550% |
| s2n_pq_kem_test | 0.05 | 23.2 | 46300% |
| s2n_record_size_test | 0.09 | 23.39 | 25889% |
We should determine why the kTLS job is sometimes running some of the tests very slowly. It might be related to how qemu is configured.