Get compiled script directory via sys.executable returns 8.3 filenames (DOS SFN)
Neo23x0 opened this issue · comments
I try to get the current directory name of the compiled script by the use of the following statement:
application_path = os.path.dirname(os.path.realpath(sys.executable))
When I run the python script it returns:
_C:\Users\neo\Downloads\loki-master_
When I run the compiled executable, it returns:
_C:\Users\neo\DOWNLO1\LOKI-M1_
This is the old MS DOS 8.3 filename style called SFN. I use the application path to exclude it from a directory walk later on but the path returned by os.walk() is obviously different than the one returned by sys.executable.
Is this a bug? Is there a way to retrieve the executable's full path?
I want to allow users to run the executable from other working directories.
I fixed the problem myself by the use of win32api:
dos_path = os.path.dirname(os.path.realpath(sys.executable))
application_path = win32api.GetLongPathName(dos_path)
Could this be added to the doc? Where?
Yes, this should be documented as a difference between running in normal Python and running under PyInstaller. PYI explicitly uses GetShortPathName to set the value of sys.executable
with an 8.3 format path as its directory component - "to overcome the Python and PyInstaller limitation to run with foreign characters in directory names." (pyi_path.c:157
)
In normal Python, sys.executable
contains the full path to argv[0]
. On Windows, argv[0] is encoded using an MS-DOS codepage. Renaming python.exe to something like pyxěščřžýáíé.exe
(or putting it in a folder with that name) will result in a worthless sys.executable
and the only way to get a good one is with the Win32 API GetCommandLineW for getting the command line as wide characters (i.e. UTF-16 encoded).
However, there is a bug(?) in pywin32
where there is no way to call GetCommandLineW. Calling win32api.GetCommandLine
always calls the ANSI variant which returns the program path encoded with a codepage - the same thing you get from sys.executable
in normal Python. So you actually have to go through ctypes
to call GetCommandLineW.
I want to be able to change sys.executable
to always return a usable filename, but this isn't possible on Windows with Python 2.7. sys.executable
is required to be a bytes
type, so we can't just change it to a unicode
. Also, python's filesystem functions will use the A variants of Windows APIs - which expect a codepage encoding (mbcs
) whenever they are passed a bytes
type, so we can't change sys.executable
to be UTF-16 encoded either.
(On Python 3.3, sys.executable
is a unicode
type, but I didn't check if it actually contains valid data for the odd filename above.)
It looks like the current behavior of using an MS-DOS 8.3 ShortFileName is the best we can do for now. It has one shortcoming though. The path to the .exe's folder is encoded as an SFN, but the filename of the .exe itself is not - it's codepage-encoded as it comes from argv.
I kind of want to go through all of the bootloader code and add hungarian-notation to all of the string variables to specify the encoding - workpath_u8
, argv_mbcs
and so on.
I'd prefer to have this fixed in pyi_path.c
. But as I read your explanation I have the impression that we can't get this fixed as long as we are supporting Python 2.x Is this correct?
we can't get this fixed as long as we are supporting Python 2.x
Yes. sys.executable
is broken, for most non-ASCII characters, on Python 2.x for Windows, even for vanilla Python without PyInstaller.
PyI's workaround of using a ShortFileName instead of encoding with mbcs
improves support over vanilla Python's, but raises this issue of not being able to compare filenames easily. It is also not available for some filesystems, especially SMB network shares. This difference is responsible for the paths I observed here and may be responsible for that issue in some way also, as he notes that the issue only happens on D:\ and not C:. It also needs to be documented as it deviates from vanilla Python's behavior.
Besides, shouldn't you be using sys._MEIPASS
instead of dirname(sys.executable)
(or __file__
etc) to find data-files when your app is intended to be frozen?
I wonder what the encoding for _MEIPASS
is. It's probably UTF-8, but if it's a bytes
type on python2 it will run into the snag of the Windows filesystem functions using the ANSI variants and trying to interpret it as mbcs
encoded. Maybe _MEIPASS
could be changed to a unicode
type?
That's what my code looks like. (still clumsy but working)
if getattr(sys, 'frozen', False):
application_path = os.path.dirname(sys.executable)
elif __file__:
application_path = os.path.dirname(__file__)
# unknown if still necessary
if application_path == "":
application_path = os.path.dirname(os.path.realpath(__file__))
if "~" in application_path:
application_path = win32api.GetLongPathName(application_path)
As codewarrior wrote, you must use _MEIPASS
, see https://pythonhosted.org/PyInstaller/#adapting-to-being-frozen
Doesn't work for me. I build my file in one file mode (-F). Using _MAIPASS I get the path of the extracted archive contents but not the path where the executable resides.
C:\Users\neo\AppData\Local\Temp
Oh, you're actually using the app by putting it in a folder where it is going to work. Then what you're doing with GetLongPathName will do what you want, but with the caveat that it won't work on filesystems that don't support 8.3 filenames - you'd have to go directly to GetCommandLineW (as I noted a few comments ago, or perhaps GetModuleFileNameW?) if you need that to work.