Update Python to Version 3.12
skargbo opened this issue · comments
It appears that the current Python version used in the MSC project is vulnerable to a number of high severity security issues that have been disclosed to the public. To address these vulnerabilities, I propose bumping the Python version 3.12, which includes patches for these CVEs.
The following CVEs have been identified in the current version 3.10 of the Python framework used by MSC
This is more difficult than you might think, due to some breakage in the Mac builds of the Python frameworks, making relocation difficult. If anyone wants to dig in and figure the issues out, I'd appreciate the help.
Specifically: If I just use the existing build scripts (but update the Python version to 3.12.2), the Python framework is subtly broken:
$ munki-python
Could not find platform dependent libraries <exec_prefix>
Python 3.12.2 (v3.12.2:6abddd9f6a, Feb 6 2024, 17:02:06) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ managedsoftwareupdate
Could not find platform dependent libraries <exec_prefix>
Traceback (most recent call last):
File "/usr/local/munki/.managedsoftwareupdate.py", line 30, in <module>
import subprocess
File "/usr/local/munki/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 104, in <module>
from _posixsubprocess import fork_exec as _fork_exec
ModuleNotFoundError: No module named '_posixsubprocess'
This is almost certainly the same issue as this: gregneagle/relocatable-python#31 and is why munki-python has remained on 3.10.x for so long.
Once you work around the issue with a shell script replacing /usr/local/munki/munki-python (which means yet another layer of wrapping and redirection, and yet another process layer), you find that Munki makes use of at least one standard lib module that's been removed in 3.12:
$ managedsoftwareupdate
Traceback (most recent call last):
File "/usr/local/munki/.managedsoftwareupdate.py", line 63, in <module>
from munkilib import appleupdates
File "/usr/local/munki/munkilib/appleupdates/__init__.py", line 5, in <module>
from .core import *
File "/usr/local/munki/munkilib/appleupdates/core.py", line 23, in <module>
from . import au
File "/usr/local/munki/munkilib/appleupdates/au.py", line 43, in <module>
from . import sync
File "/usr/local/munki/munkilib/appleupdates/sync.py", line 56, in <module>
from .. import fetch
File "/usr/local/munki/munkilib/fetch.py", line 27, in <module>
import imp
ModuleNotFoundError: No module named 'imp'
The imp module is used to import middleware and repo-plugins. So we'll need to figure out the current replacement for this.
imp fix is here: 1850dbe
I posted a potential workaround for gregneagle/relocatable-python#31 (comment).
What if instead of relying on a symlink, we just incorporated the tiny shell script and use that for invoking Python?
It could even be signed, in order to ensure compliance with macOS PPPC expectations for running Python interpreters (have not tested that at all).
FWIW we have discovered that VMware's embedded Munki is using 3.11 so it begs the question - what did they do, why did they do it, and why didn't they send a PR to help the community.
Betting they are just using the actual/normalized path to the actual Python 3.11 binary as the shebang, which avoids the platform libraries issue.
$ /usr/local/hub/python3 -V
Python 3.11.6
$ /usr/local/hub/python3
Python 3.11.6 (tags/v3.11.6-dirty:8b6ee5ba3b, Nov 23 2023, 18:37:49) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> exit()
$ file /usr/local/hub/python3
/usr/local/hub/python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
/usr/local/hub/python3 (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/local/hub/python3 (for architecture arm64): Mach-O 64-bit executable arm64
So it looks like they went the route of a compiled binary shim...
$ strings /usr/local/hub/python3
realpath: %s
__PYVENV_LAUNCHER__
posix_spawn: %s
Resources/Python.app/Contents/MacOS/Python
posix_spawnattr_int
posix_spawnattr_setbinpref
posix_spawnattr_setbinpref failed to copy
posix_spawnattr_setflags
realpath: %s
__PYVENV_LAUNCHER__
posix_spawn: %s
Resources/Python.app/Contents/MacOS/Python
posix_spawnattr_int
posix_spawnattr_setbinpref
posix_spawnattr_setbinpref failed to copy
posix_spawnattr_setflags
I wish I had contacts over there still. It really sucks that they solved this and didn't help the community.
Most of the needed work is done as of this commit: 9e541a8
Beta release here: https://github.com/munki/munki/releases/tag/v6.5.0b1
we have some installcheck_scripts using the distutils module (https://github.com/autopkg/dataJAR-recipes/blob/master/Firefox-locale/Firefox.munki.recipe#L109.. with distutils being removed with Python 3.12, what's the recommended path to overcome this?
I'm hoping/guessing that we can import another module to use in it's place?
Looking at a8494ab I guess we can:
from munkilib.pkgutils import MunkiLooseVersion as LooseVersion and on an import error revert to from distutils.version import LooseVersion?
So something like:
try:
from munkilib.pkgutils import MunkiLooseVersion as LooseVersion
except ModuleNotFoundError:
from distutils.version import LooseVersion
Python moves forward and deprecates and removes things. If your scripts use things that have been removed, yes, you must adapt.
@gregneagle true!
Can MunkiLooseVersion be seen as a drop in replacement for LooseVersion, for simple comparisons?
Just wanted to clarify as will then test and move to that.
I would think so, but there are slight differences in behavior (which is why MunkiLooseVersion was created)
Munki 6.5 has been released, and includes Python 3.12.2.