Bug) accept() syscall returned fd > 7 will unintentionally end the hooked application

[Jay Jang]

Hello, thank you for sharing the code of libdft64.

I just wanted to inform an interesting bug with this line of code with accept system call handling part of the code:

case __NR_accept:
case __NR_accept4:
	/* not successful; optimized branch */
	if (unlikely((long)ctx->ret < 0))
		return;
	       /*
		* if the socket argument is interesting,
		* the returned handle of accept(2) is also
		* interesting
		*/
	if (likely(fdset.find(args[SYSCALL_ARG0]) !=
				fdset.end()))
		/* add the descriptor to the monitored set */
		fdset.insert((int)ctx->ret);

It is stated in the manual: https://man7.org/linux/man-pages/man2/accept.2.html that accept() returns a new file descriptor referring to that socket, and whenever (int)ctx->ret value is greater than 7, the hooked program abruptly ends. This is a key part because there is no segmentation fault or anything of an error, but just that the program gracefully exits.

The output looks like this:

pin-3.20_build/pin -follow-execv -t /home/lib/libdft-dta.so -- ./redis-server
...
210032:M 23 Mar 2023 17:53:44.985 * Increased maximum number of open files to 10032 (it was originally set to 1024).
        ► socket(2) fd add 6
        ► socket(2) fd add 7
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 5.0.4 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 210032
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

read(2) fd: 8
read(2) taint clear
210032:M 23 Mar 2023 17:53:46.708 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in ord
er to retain the setting after a reboot. Redis must be restarted after THP is disabled.
210032:M 23 Mar 2023 17:53:46.720 * Ready to accept connections
...

and then on the other terminal, if I were to execute ./redis-cli, then the above screen will output

...
read(2) fd: 8
read(2) taint clear
fd add accept
read(2) fd: 9
read(2) taint clear
read(2) fd: 8

and the program ends with the connection closed on the other terminal.

I have statically replaced (int)ctx->ret with a value less than 8, and was able to successfully determine that the file descriptor number is the issue. I have also tested using std::list and std::hash_set instead of

/* set of interesting descriptors (sockets) */
static set<int> fdset;

Doesn't seem to fix it though... Also tried GDBing the libdft-dta.so, but nothing useful came out of there either.

[sp1npx]

I reviewed the code of libdft-dta.cpp, and it appears that it won't exit due to the file descriptor values. Could you try reproducing the issue with a minimal program such as a simple server/client instead of using Redis? Additionally, DTA will only exit with an 'alert' and log something. Can you check if there are any log files there?

[Jay Jang]

Hello, thank you for your response. So I did some debug testing as you suggested with a minimum server program like https://gist.github.com/GaryMeloney/424e502956f49b2fc58487d9d510ed80, and I realized that I couldn't reproduce the issue. Therefore, I tried building everything from scratch with a very clean slate, and it seemed like it might have been some other issues outside of libdft. I apologize for the inconvenience! Thank you again for sharing the code.