`select`ing a `forkpty`'ed master file descriptor does not return when the child process exits

[Nathan Wang]

Description

I think this is related to #9333.

If I:

1. Call forkpty()
2. In the child, wait a bit and exit
3. In the parent, select() to read data from the master file descriptor

The select() call will not return even after the child process exits, causing the program to hang indefinitely. In runc (and normal ubuntu), I believe the select() function call will return with the master file descriptor set, and attempting to read from the file descriptor will either return 0 bytes read or return an error (I forgot which).

I'm not sure what the "correct" behavior for what should happen is if a select()ed file descriptor is closed during the select() call; it seems like it's possible that this is undefined behavior: https://stackoverflow.com/questions/543541/what-does-select2-do-if-you-close2-a-file-descriptor-in-a-separate-thread Regardless, it seems like a nonzero number of programs rely on select() exiting when the file descriptor is closed, notably pty.spawn() in python.

As a workaround, the parent process can listen for SIGCHILD, which will cause select() to return with an error when the child exits.

I think this might be related to why #9333 hangs -- I believe pty.spawn() internally executes a select() function call on the master file descriptor that never returns.

Steps to reproduce

Run this file in runsc:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <pty.h>
#include <utmp.h>
#include <sys/select.h>
#include <termios.h>

int main(int argc, char *argv[]) {
  int master_fd;

  // Fork and create a new PTY
  pid_t pid = forkpty(&master_fd, NULL, NULL, NULL);
  if (pid < 0) {
    perror("pid");
    exit(1);
  }

  if (pid == 0) {
    usleep(1000 * 50); // 50 ms
    exit(0);
  } else {
    // parent
    fd_set read_fds;
    FD_ZERO(&read_fds);
    FD_SET(master_fd, &read_fds);
    int max_fd = master_fd + 1;

    struct timeval tv;
    int retval;

    // Set timeout to 0.2 seconds
    tv.tv_sec = 0;
    tv.tv_usec = 200000;

    // Wait for data on either the PTY or stdin.
    retval = select(max_fd, &read_fds, NULL, NULL, &tv);

    if (retval < 0) {
      perror("select");
    }

    if (FD_ISSET(master_fd, &read_fds)) {
      printf("success!\n");
    } else {
      printf("fail\n");
    }
  }
}

Compile with gcc -o test test.c -lutil. The expected behavior is that select() will return once the child process exits and master_fd will be set, but the current behavior is that select() never returns until the timeout is triggered.

runsc version

No response

docker version (if using docker)

Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:08:01 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:08:01 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.26
  GitCommit:        3dd1e886e55dd695541fdcd67420c2888645a495
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

uname

Linux ip-10-1-5-239 5.15.0-1048-aws #53~20.04.1-Ubuntu SMP Wed Oct 4 16:44:20 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

kubectl (if using Kubernetes)

No response

repo state (if built from source)

release-20240122.0-26-g9b9d06b10

runsc debug logs (if available)

No response

[Kevin Krakauer]

The Linux behavior can be tested via a syscall test.

@ayushr2 do you know why select wouldn't return? Do we have to do something in devpts to pipe a notification up to select?

[Ayush Ranjan]

Do we have to do something in devpts to pipe a notification up to select?

Briefly looking at select(2) code, it could also be a race between checking for open-ness of FD (pkg/sentry/syscalls/linux/sys_poll.go:doSelect() => t.GetFile(fd)) and registering notifications (pkg/sentry/syscalls/linux/sys_poll.go:pollBlock() => initReadiness()).

Need to take a closer look...

[Ayush Ranjan]

We were missing a notification when all replicas are closed. Should be fixed in #9979.