Test failure with OpenSSL 3.1.0

[Nobuyoshi Nakada]

In OpenSSL::TestSSL#test_connect_certificate_verify_failed_exception_message,

[1/0] OpenSSL::TestSSL#test_connect_certificate_verify_failed_exception_message = 0.03 s
  1) Failure:
OpenSSL::TestSSL#test_connect_certificate_verify_failed_exception_message [/Users/nobu/src/ruby/master/src/test/openssl/utils.rb:295]:
exceptions on 1 threads:
#<Thread:0x000000010d0f8d00 /Users/nobu/src/ruby/master/src/test/openssl/utils.rb:269 dead>:
/Users/nobu/src/ruby/master/src/tool/lib/test/unit/assertions.rb:109:in `assert': Expected Exception(OpenSSL::SSL::SSLError) was raised, but the message doesn't match. (Test::Unit::AssertionFailedError)
Expected /self.signed/ to match "SSL_connect SYSCALL returned=5 errno=0 peeraddr=127.0.0.1:53969 state=error: certificate verify failed".
	from /Users/nobu/src/ruby/master/src/tool/lib/core_assertions.rb:495:in `assert'
	from /Users/nobu/src/ruby/master/src/tool/lib/core_assertions.rb:466:in `assert_raise_with_message'
	from /Users/nobu/src/ruby/master/src/test/openssl/test_ssl.rb:1051:in `block in test_connect_certificate_verify_failed_exception_message'
	from /Users/nobu/src/ruby/master/src/test/openssl/utils.rb:273:in `block (2 levels) in start_server'

Seems like SSL_R_TLSV1_ALERT_UNKNOWN_CA is returned instead of SSL_R_CERTIFICATE_VERIFY_FAILED?

[Kazuki Yamaguchi]

I haven't been able to reproduce this locally with OpenSSL 3.1.0 on Linux. The message contains certificate verify failed (self-signed certificate in certificate chain) as expected.

The test case OpenSSL::TestSSL#test_connect_certificate_verify_failed_exception_message does want to check the /self.signed/ part because it was introduced by #99, which tried to add some context as to why the certificate verification failed.

Seems like SSL_R_TLSV1_ALERT_UNKNOWN_CA is returned instead of SSL_R_CERTIFICATE_VERIFY_FAILED?

SSL_R_TLSV1_ALERT_UNKNOWN_CA is a server-side error (server receiving an alert). Since an error queue is created for each native thread, it should not get mixed with client-side errors.

[Kazuki Yamaguchi]

Expected /self.signed/ to match "SSL_connect SYSCALL returned=5 errno=0 peeraddr=127.0.0.1:53969 state=error: certificate verify failed".

SSL_get_error() returned SSL_ERROR_SYSCALL instead of SSL_ERROR_SSL, but with errno == 0. This looks strange.

[Kazuki Yamaguchi]

The man page of SSL_get_error() says:

       SSL_ERROR_SYSCALL
           Some non-recoverable, fatal I/O error occurred. The OpenSSL error queue may contain more information on the error. For socket I/O on Unix
           systems, consult errno for details. If this error occurs then no further I/O operations should be performed on the connection and
           SSL_shutdown() must not be called.

           This value can also be returned for other errors, check the error queue for details.

       SSL_ERROR_SSL
           A non-recoverable, fatal error in the SSL library occurred, usually a protocol error.  The OpenSSL error queue contains more information on
           the error. If this error occurs then no further I/O operations should be performed on the connection and SSL_shutdown() must not be called.

Also, Google search "SSL_connect SYSCALL returned=5 errno=0 state=error: certificate verify failed" gives many hits of articles written several years ago; this may not be new in OpenSSL 3.1.0.

#640 should fix this... but since I haven't reproduced the error in my local environment, it's not been tested.