Tests fail on ppc64el

[Evan Broder]

The tty-steal test on ppc64el fails with the following logs:

Unable to find `prctl.set_ptracer`, skipping `PR_SET_PTRACER`.
spawned children: me=14648 victim=14649 reptyr=14650
world
[+] Making sure we have permission to attach...
[+] session leader of pid 14649 = 14649
[+] found terminal emulator process: 14648
[+] Listening on socket: /tmp/reptyr.VhiRVA/reptyr.sock
[+] Attaching terminal emulator pid=14648
[+] Allocated scratch page: 7fffacce0000
[+] Attached to terminal emulator (pid 14648)
[+] Checking fd: 0: st_dev=8800
[+] Checking fd: 1: st_dev=8800
[+] Checking fd: 2: st_dev=8800
[+] Checking fd: 5: st_dev=502
[+] found a ptmx fd: 5
[+] found a master fd: 5
[+] Checking fd: 6: st_dev=502
[+] found a ptmx fd: 6
[+] Opened fd 94 in the child.
[+] Connected to the shared socket.
[+] Sent the pty fd, going to receive it.
[-] Error receiving message.
Unable to attach to pid 14649: Resource temporarily unavailable
world
[...]

I've discussed this with @nelhage in person but since I have access to a ppc64el machine now, opening a ticket to track the debugging work. I'm currently trying to understand why the powerpc implementation of arch_set_syscall is using PTRACE_POKEUSER, since AIUI the syscall number goes in a register, not a memory address.

[Evan Broder]

OK that was a red herring - ptrace has interfaces through PEEKUSER and POKEUSER for interacting with registers (in addition to GETREGSET and SETREGSET.

I'm currently suspicious that the return value from a syscall ends up in the result field of struct pt_regs, not in gpr[3]. But if I make that change, I get a new, earlier error:

[+] Checking fd: 6: st_dev=502
[+] found a ptmx fd: 6
Unable to attach to pid 1954: Socket type not supported

which implies that socket(AF_UNIX, SOCK_DGRAM, 0) is failing with ESOCKTNOSUPPORT which makes no sense.

[Nelson Elhage]

I find straceing the reptyr process pretty helpful for debugging weird issues, since so much of what it does is ~directly at the syscall layer.

You can probably modify

reptyr/test/tty-steal.py

Line 33 in e78f1f4

 reptyr = pexpect.spawn("./reptyr -V -T %d" % (child.pid,)) 

to add strace -o reptyr.strace or something and then pore over that.

[Evan Broder]

Yeah I did try something similar earlier, but it doesn't let me see the registers that are being poked into place:

ptrace(PTRACE_GETREGSET, 20911, NT_PRSTATUS, [{iov_base=0x7fffe1e014d8, iov_len=352}]) = 0
ptrace(PTRACE_SETREGSET, 20911, NT_PRSTATUS, [{iov_base=0x7fffe1e014d8, iov_len=352}]) = 0
ptrace(PTRACE_SETREGSET, 20911, NT_PRSTATUS, [{iov_base=0x7fffe1e02848, iov_len=352}]) = 0
ptrace(PTRACE_POKEUSER, 20911, r0, 0x118) = 0
ptrace(PTRACE_DETACH, 20911, NULL, 0)   = 0
close(3)                                = 0
unlink("/tmp/reptyr.ugBArk/reptyr.sock") = 0
rmdir("/tmp/reptyr.ugBArk")             = 0
write(2, "Unable to attach to pid 20924: S"..., 57Unable to attach to pid 20924: Socket type not supported

Having thought about it more, though, I do think that the socket call is actually returning with ESOCKTNOSUPPORT. The victim process only has ~6 fds open, so this line from the original output stream does actually make no sense:

[+] Opened fd 94 in the child.

Still not sure why, though, or what arguments are actually getting fed into the syscall.

[Evan Broder]

Aha:

(gdb) p arch_syscall_numbers[0].nr_socket
$5 = 0

[Evan Broder]

powerpc seems to have both a socketcall syscall and the direct socket etc. syscalls, but because socketcall exists, it was going through this logic:

reptyr/platform/linux/linux.c

Lines 415 to 418 in e78f1f4

 // We assume that socketcall is only used on 32-bit 
 // architectures. If there are any 64-bit architectures that do 
 // socketcall, and we port to them, this will need to change. 
 uint32_t args[] = {p0, p1, p2, p3, p4}; 

which, as promised, does not seem to work on 64-bit architectures. Swapping the precedents in default-syscalls.h (to prefer socket over socketcall unless __NR_socket is undefined) seems to work. I'll send a patch in a sec.