Some interpose hooks are skipped on Big Sur
qiuwenqiao opened this issue · comments
Last commit: https://github.com/steven-michaud/HookCase/issues/21
I tested new version (5.0.1), and the problem still exists. I try to write a simple case.
INTERPOSE_FUNCTION(open)
static int Hooked_open(const char * filename, int flags, mode_t mode)
{
const char * lpDot = strrchr(filename, '.');
if (lpDot && (strcmp(lpDot, ".pdf")==0))
{
printf("open %s\n", filename);
}
return open(filename, flags, mode);
}
Then, Run Preview and open a pdf file. In macos 10.15.7, It will show 4 records. But In macos 11.1 It only show 1 record. So, I think maybe some hooks didn't attach.
The problem I see is worse that what you report. Using an interpose hook (as you do), I don't see any calls to open() on macOS 11.2.1. They do appear if you use a patch hook. But I need to figure out why the interpose hooks aren't working.
This is a bug in HookCase. There's a structure called a lazy pointer table that HookCase uses to support interpose hooks. In the past it was always located in the __DATA segment. But now, in Big Sur, it's sometimes located in the __DATA_CONST segment. I need to change HookCase code so that it will look in both places (if need be).
I've written a patch, and will be testing it over the next few days. Once I'm reasonably sure it works properly, I'll land it here and bump the version number. Then I'll ask you to test it.
This bug should now be fixed. Please comment here if you still see the exact same problem you originally reported. Please open a new bug if you see some other problem.
One more thing I should mention. HookCase can have trouble with variable argument functions like open(). This function only has one possible "extra" argument -- mode_t mode. So your hook needs to always specify the mode argument, even for cases in which it isn't present. This you've done correctly.
But there's another problem: The AMD64 processor ABI specifies that, on calling a function with variable arguments, the RAX register should specify the number of arguments stored in "vector registers" (XMM0 to XMM7). Since mode is an integer, RAX should always be 0 on calls to open(). The only way to guarantee this in a HookCase hook is to put the call to the original open() function at the very beginning of the hook. For example:
static int Hooked_open(const char *path, int oflag, mode_t mode)
{
int retval = open(path, oflag, mode);
const char *lpDot = NULL;
if (path) {
lpDot = strrchr(path, '.');
}
if (lpDot && (strcmp(lpDot, ".pdf")==0)) {
LogWithFormat(true, "Hook.mm: open(): path %s, oflag 0x%x, mode 0x%x, returning %i",
path, oflag, mode, retval);
PrintStackTrace();
}
return retval;
}
Problems with variable argument functions are probably present in all "function hooking" software. printf() is even worse, because it can take almost any number of arguments. My advice is to avoid hooking variable argument functions. Instead hook their va_list variants, like vprintf(). A variable argument function will almost always call some kind of va_list variant. But to hook it you'll need to use a patch hook (since the high-level code may not call it directly).
I have tested in 10.15.7/11.1/11.2.2, unfortunately, The problem still exists, and have the same logs. by the way, should I modify the library template?
I'll talk only about your results on macOS 11, since (according to your original report) that's where the problem exists.
Are you sure you're using HookCase version 5.0.2? What version number do you see when you do kextstat -b org.smichaud.HookCase?
Please try testing with a hook library that contains only the hook from my previous comment.
Also make sure you're recompiling HookCase.kext and your hook libraries on every different version of macOS that you test on.
I assume you're still testing with Preview, by opening a file in it.
I retested and found some interesting things. Here are test code.
static int Hooked_open(const char *path, int oflag, mode_t mode)
{
int retval = open(path, oflag, mode);
const char *lpDot = NULL;
if (path) {
lpDot = strrchr(path, '.');
}
if (lpDot && ((strcmp(lpDot, ".pdf")==0) || (strcmp(lpDot, ".jpg")==0)) ) {
LogWithFormat(true, "Hook.mm: open(): path %s, oflag 0x%x, mode 0x%x, returning %i",
path, oflag, mode, retval);
// PrintStackTrace();
}
return retval;
}
First of all. I comfired the hookcase version.
kextstat -b org.smichaud.HookCase
No variant specified, falling back to release
Index Refs Address Size Wired Name (Version) UUID
160 0 0xffffff7fa2414000 0x1a000 0x1a000 org.smichaud.HookCase (5.0.2) 4B2D06F9-8DEC-3846-80F9-E773408914A7 <8 6 5 3 2 1>
Then, I Run
HC_INSERT_LIBRARY=/Users/mac/Downloads/hook.dylib /System/Applications/Preview.app/Contents/MacOS/Preview
When I opened a jpg file, The result is correct.
_(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x700006e7f000] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x8000, mode 0x4c1a, returning 4
(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x8100, mode 0xe21a, returning 4
(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x0, mode 0x0, returning 4
(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x0, mode 0xffe0, returning 4
(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x0, mode 0x0, returning 4
(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x8004, mode 0xa, returning 4
2021-03-01 10:47:34.889 Preview[817:20886] WARNING: The SplitView is not layer-backed, but trying to use overlay sidebars.. implicitly layer-backing for now. Please file a radar against this app if you see this.
(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x0, mode 0x0, returning 4
(Mon Mar 1 10:47:34 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x0, mode 0x0, returning 4
(Mon Mar 1 10:47:35 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x0, mode 0x0, returning 4
(Mon Mar 1 10:47:35 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x0, mode 0xffe0, returning 6
(Mon Mar 1 10:47:36 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x70000710e000] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x2, mode 0xcf1a, returning 6
2021-03-01 10:47:36.589 Preview[817:20886] +[NSSavePanel warmUp] attempted warmup
(Mon Mar 1 10:47:40 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x700007191000] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x2, mode 0xb408, returning 6
(Mon Mar 1 10:47:40 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x70000708b000] Hook.mm: open(): path /Users/mac/Downloads/5.jpg, oflag 0x2, mode 0x9f1a, returning 11
But When I opened a pdf file. I found that the path is null
(Mon Mar 1 10:48:12 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x700006e7f000] Hook.mm: open(): path , oflag 0x8000, mode 0xe026, returning 13
(Mon Mar 1 10:48:12 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path , oflag 0x8100, mode 0x4826, returning 13
(Mon Mar 1 10:48:12 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path , oflag 0x0, mode 0xffe0, returning 13
(Mon Mar 1 10:48:12 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path , oflag 0x0, mode 0x0, returning 13
(Mon Mar 1 10:48:12 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path , oflag 0x8004, mode 0xb690, returning 13
2021-03-01 10:48:12.457 Preview[817:20886] WARNING: The SplitView is not layer-backed, but trying to use overlay sidebars.. implicitly layer-backing for now. Please file a radar against this app if you see this.
(Mon Mar 1 10:48:13 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x117b69e00] Hook.mm: open(): path , oflag 0x0, mode 0xffe0, returning 16
(Mon Mar 1 10:48:13 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x700006d79000] Hook.mm: open(): path , oflag 0x2, mode 0x7f26, returning 16
(Mon Mar 1 10:48:17 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x700006cf6000] Hook.mm: open(): path , oflag 0x2, mode 0xefe0, returning 16
(Mon Mar 1 10:48:17 2021) /System/Applications/Preview.app/Contents/MacOS/Preview[817] [0x700006d79000] Hook.mm: open(): path , oflag 0x2, mode 0x7f26, returning 16
Interesting indeed.
It looks like the calls from Preview to open .PDF files succeeded. I assume the file you tried to open did load successfully.
path wasn't NULL. In that case the hook would have crashed. Instead I suspect it started with an improperly formatted multibyte character -- one that wasn't labelled as such in the character stream. The first byte of some multibyte characters is 0. So if the logger was only expecting a single byte character, it might have thought that path pointed to an empty array of characters.
Exactly what path were you trying to load? I assume it contained some Chinese characters, or at least some Roman characters from the 简体中文 or 繁體中文 character sets.
Yes, I do speak a little Chinese :-)
Another possibility is that path contained a correctly formatted multibyte (non-UTF8) character stream whose very first byte was 0. Both printf() and my interpose library template's LogWithFormatV() function would assume the character stream is UTF8, and therefore interpret the first character as a terminal NULL.
If this is the case, I'm not sure what I can do about it, if anything.
OK, I've now figured it out:
If your preferred language (偏好语言) is Chinese (simplified or traditional) and the path you're trying to open contains at least one Chinese character (simplified or traditional), path will be empty in the open() hook.
This is actually a completely different bug from what was originally reported (as I understood it), and what I fixed with 0f843c2. It's also completely unrelated. In the next few days I'll open a new issue on this bug. But like I said, I'm not sure there's anything I can do about it.
I think probably the terminal does not support Chinese characters. Because
for (int i=0; i<strlen(path);i++)
{
printf("%x", path[i]);
}
The result is correct.
I tested the code with english pathname again. seems the interpose hooks bug is fixed.
As you say, printf() works just fine whether your preferred language is English or Chinese, and whether or not path contains Chinese characters. And path doesn't contain any embedded NULLs even when your preferred language is Chinese. So that's not the reason my code in LogWithFormatV() can't read it.
I need to figure out why printf() works but my code doesn't. Once I've done that, I'll open a new issue and fix the bug.
It may be a while, though. I'm having to dig very deep into undocumented CoreFoundation code.
I just opened Issue #25 and landed a fix for it.