Open control FD in gofer server (and directfs gofer client) with O_PATH.

[Ayush Ranjan]

Description

          > mountpoint_s3::fuse: open failed: inode error: inode 3 (full key "synmon/test_file.txt") is not writable while being read

The issue is that gVisor's gofer client (pkg/fsimpl/gofer/) opens the file read-only first and then opens it for writing, but mountpoint_s3::fuse refuses to support that. The gofer client first caches the dentry for the file being opened. It holds a "control FD" for all the dentries it caches. This control FD is opened with O_RDONLY|O_NONBLOCK:

gvisor/pkg/sentry/fsimpl/gofer/directfs_dentry.go

Lines 41 to 50 in f67e10c

 // tryOpen tries to open() with different modes in the following order: 
 //  1. RDONLY | NONBLOCK: for all files, directories, ro mounts, FIFOs. 
 //     Use non-blocking to prevent getting stuck inside open(2) for 
 //     FIFOs. This option has no effect on regular files. 
 //  2. PATH: for symlinks, sockets. 
 func tryOpen(open func(int) (int, error)) (int, error) { 
 	flags := []int{ 
 		unix.O_RDONLY | unix.O_NONBLOCK, 
 		unix.O_PATH, 
 	} 

I think we should change that logic to open regular files with O_PATH only. Directories and FIFOs can continue to be opened with O_RDONLY.

Originally posted by @ayushr2 in awslabs/mountpoint-s3#862 (comment)

[Ayush Ranjan]

This work may be more involved. The control FD is used to perform various operations on a file (see usages). Some of those operations will fail with EBADF if the control FD is a O_PATH FD. For instance: fchmod(2), fchown(2) and fgetxattr(2). See Linux kernel source code for these syscalls; they use fdget(fd) => __fget_light(fd, FMODE_PATH) which returns NULL for O_PATH FDs. So we will need to open all files with O_PATH and upgrade to a readable FD when needed OR use f*at(2) syscall variant with AT_EMPTY_PATH (which effectively bypasses this issue).

Some questions:

• How is fchmod(2) working for symlinks?
  

  gvisor/pkg/sentry/fsimpl/gofer/directfs_dentry.go

  Lines 268 to 271 in 6e61813

   func (d *directfsDentry) chmod(ctx context.Context, mode uint16) error { 
   	if !d.isSocket() { 
   		return unix.Fchmod(d.controlFD, uint32(mode)) 
   	} 

• How is fgetxattr(2) working for symlinks and sockets?
  

  gvisor/pkg/sentry/fsimpl/gofer/directfs_dentry.go

  Lines 426 to 430 in 6e61813

   func (d *directfsDentry) getXattr(name string, size uint64) (string, error) { 
   	data := make([]byte, size) 
   	if _, err := unix.Fgetxattr(d.controlFD, name, data); err != nil { 
   		return "", err 
   	} 

• Can we open FIFOs with O_PATH?