Not working with PyInstaller

[vdun]

# cat h.py
print('hello')
# pyinstaller -F -c h.py
...
# ulexecve ./dist/h
[1512371] Cannot open PyInstaller archive from executable (/usr/bin/python3.10) or external archive (/usr/bin/python3.10.pkg)

[Vincent Berg]

Quickly debugging this makes it look like pyinstaller has a bootloader that resolves the calling binary (that it tries to extract the pydata segment from) via /proc/self/exe as can be seen in pyi_path.c.

The --fallback method that uses memfd_create() also will not work for the same reasons. Output looks like:

$ ulexecve --fallback ./dist/h
[3051] Cannot open PyInstaller archive from executable (/memfd: (deleted)) or external archive (/memfd: (deleted).pkg)

We may be able to solve this by using a prctl() call with PR_SET_MM_EXE_FILE. Based on the manpage the kernel would require us to unmap everything else first so this might be not as easy as we would hope it to be. This will also severely restrict us as it is rather likely we get an EPERM back from that call as one can only set this option if the caller has the CAP_SYS_RESOURCE capability.

I'll see if I have some time in the next few weeks to attempt to patch this but it's already clear to me that any binary produced by PyInstaller will not be as easily executable in userland due to these constraints that are hard to work around and, in some cases, simply can't be worked around.

[Vincent Berg]

There are two other ways this can work. Both however require access to a writable filesystem though.

The first option is to extract the embedded archive from the PyInstaller binary, write this all to temporary storage, and then name the binary to be executed such that we can also create the binary.pkg version. If we then set _MEIPASS2 we trick the PyInstaller binary into thinking it has already unpacked its embedded archive. This is all a lot of work, we would have to write a ton of things to temporary storage although we might be able to do it by using memfds for all files and simply only writing symlinks to disk.

The second option is less reliable in very restricted scenario's but a whole lot easier. PyInstaller uses /proc/self/exe so we can attempt to replace, braindead, in the binary /proc/self/exe with a path we control that points to the binary again and not to the python executable. Given that we don't want to reassemble the ELF and all the instructions completely the easiest way is to come up with some random path that is just as long as the string /proc/self/exe. Assuming we can write to /tmp then something like /tmp/XXXXXXXXX can work. We symlink then from that filename to the memfd created binary that is only in memory and then we can execute from there.

Both options will leave at minimum a bunch of symlinks behind and the first option probably also a binary that we copied to a different location to make it all work. So we would need to fork() as well and have a watchdog process so that after the child is done doing a userland execution to unlink the files we created.

I got a proof of concept working for the /proc/self/exe trick but I want to clean it up a bit and test it on both py2/py3 and then push it out. But so far it looks like this:

gvb@caladan:~/src/ulexecve$ cat ./tmp/dist/h  | ./ulexecve.py -
[5064] Cannot open PyInstaller archive from executable (/usr/bin/python2.7) or external archive (/usr/bin/python2.7.pkg)
gvb@caladan:~/src/ulexecve$ cat ./tmp/dist/h  | ./ulexecve.py --pyi-fallback -
hello
gvb@caladan:~/src/ulexecve$

[Vincent Berg]

The workaround as mentioned above with the /proc/self/exe trick got merged and can be used via --pyi-fallback. A new version was also published (1.3) on pypi.org.

[Vincent Berg]

@vdun Can you test if in your use cases of ulexecve the workaround trick as just released alleviates some of the pain?

I'm not a fan of implementing either one of the other options as the first one can only be used under high privilege (CAP_SYS_RESOURCE) anyway and the second is a chunk of work, very specific to PyInstaller that can break with any new upgrade at the moment.

And anyone using some form of obfuscation on a PyInstaller binary can still prevent it from being run via ulexecve. But most vanilla binaries should work fine.