mtth / hdfs

API and command line interface for HDFS

Home Page:https://hdfscli.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Python 3.12 fix for `imp.load_source` doesn't seem to work

xylar opened this issue · comments

In #, a fix was added to try to support python 3.12:
https://github.com/mtth/hdfs/blob/v2.7.1/hdfs/config.py#L23-L25
However, this function does not, in fact, seem to be in the python 3.12 version of importlib. A bit of googling turned up the following:
https://docs.python.org/3/whatsnew/3.12.html#imp
which suggests that there isn't a replacement function but the following can be added:

import importlib.util
import importlib.machinery

def load_source(modname, filename):
    loader = importlib.machinery.SourceFileLoader(modname, filename)
    spec = importlib.util.spec_from_file_location(modname, filename, loader=loader)
    module = importlib.util.module_from_spec(spec)
    # The module is always executed and not cached in sys.modules.
    # Uncomment the following line to cache the module.
    # sys.modules[module.__name__] = module
    loader.exec_module(module)
    return module

We're having trouble with hdfs with python 3.12 on conda-forge because of this:
conda-forge/python-hdfs-feedstock#28

Oh, sorry! I see that this is fixed on main and just hasn't made it into a release yet.

@xylar - FYI just released in 2.7.3.

Wonderful, thank you! I patched 2.7.1 and 2.7.2 on conda-forge when I built them but it's always nicer to have an unpatched release.