OpenSSL is thread-safe but multiple threads should not read from the same UDP socket fd concurrently
liuqun opened this issue · comments
The dtls_udp_echo server is a multi-threaded application, but things get weird when 2 server threads
are running together.
For example, I start 1 echo server, then start 2 client process:
# Echo server
/dtls_udp_echo -L 127.0.0.1 -p 23232 -v
# Then in another terminal, start 2 clients concurrently
./dtls_udp_echo -p 23232 127.0.0.1 & ./dtls_udp_echo -p 23232 127.0.0.1
I have been testing whether it could serve 2 or more clients concurrently. The tcpdump/wireshark packet sniffer shows that the DTLS handshake would be disturbed when 2 threads are running together, and only the first client can complete the DTLS handshake with the echo server.
There is a question in openssl's offical FAQ page:
- Is OpenSSL thread-safe?
Yes but with some limitations; for example, an SSL connection cannot be used concurrently by multiple threads. This is true for most OpenSSL objects.
For version 1.1.0 and later, there is nothing further you need do.
For earlier versions than 1.1.0, it is necessary for your application to set up the thread callback functions. To do this, your application must call CRYPTO_set_locking_callback(3) and one of the CRYPTO_THREADID_set... API's. See the OpenSSL threads manpage for details and "note on multi-threading" in the INSTALL file in the source distribution.
The following code does create different SSL connection for each thread, and each SSL connection has its own BIO buffer. But it doesn't work as expected when 2 clients connect to the same echo server concurrently.
int start_server()
{
fd = socket(server_addr.ss.ss_family, SOCK_DGRAM, 0);
//...
if (server_addr.ss.ss_family == AF_INET) {
bind(fd, (const struct sockaddr *) &server_addr, sizeof(struct sockaddr_in));
} else {
setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, (char *)&off, sizeof(off));
bind(fd, (const struct sockaddr *) &server_addr, sizeof(struct sockaddr_in6));
}
//...
while (1) {
memset(&client_addr, 0, sizeof(struct sockaddr_storage));
/* Create BIO */
bio = BIO_new_dgram(fd, BIO_NOCLOSE);
/* Set and activate timeouts */
timeout.tv_sec = 5;
timeout.tv_usec = 0;
BIO_ctrl(bio, BIO_CTRL_DGRAM_SET_RECV_TIMEOUT, 0, &timeout);
ssl = SSL_new(ctx);
SSL_set_bio(ssl, bio, bio);
SSL_set_options(ssl, SSL_OP_COOKIE_EXCHANGE);
while (DTLSv1_listen(ssl, (BIO_ADDR *) &client_addr) <= 0){
;
}
info = (struct pass_info*) malloc (sizeof(struct pass_info));
memcpy(&info->server_addr, &server_addr, sizeof(struct sockaddr_storage));
memcpy(&info->client_addr, &client_addr, sizeof(struct sockaddr_storage));
info->ssl = ssl;
if (pthread_create( &tid, NULL, connection_handle, info) != 0) {
perror("pthread_create");
exit(-1);
}
}
}
Here in the main loop of start_server(), all the bios share the same UDP socket fd to listen, even if each server thread has its own bio and ssl object.
In each service thread, connection_handle() builds a different socket fd and and update the bio to use the new fd.
This may be the same bug as issue 8
DTLSv1_Listen() always returns 1 when 2 or more client connect to the dtls_udp_server
