[BUG] spark magic cannot be installed on python3.11
andrew0411 opened this issue · comments
Describe the bug
On my conda environment where python version is 3.11 that uses pip 24.0, cannot be installed.
To Reproduce
conda create -n test python=3.11
conda activate test
pip install sparkmagic
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [25 lines of output]
/bin/sh: 1: krb5-config: not found
Traceback (most recent call last):
File "/home/jaehwanlee/anaconda3/envs/test/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/jaehwanlee/anaconda3/envs/test/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jaehwanlee/anaconda3/envs/test/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-gswj1ixw/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-gswj1ixw/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-gswj1ixw/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
exec(code, locals())
File "<string>", line 109, in <module>
File "<string>", line 22, in get_output
File "/home/jaehwanlee/anaconda3/envs/test/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jaehwanlee/anaconda3/envs/test/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'krb5-config --libs gssapi' returned non-zero exit status 127.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Additional context
I've tested on the environment with python=3.10 and it worked.
Hello, @andrew0411! I'm here to help you with any questions, bugs, or contributions you might have.
To resolve the issue of being unable to install sparkmagic
on Python 3.11 due to an error related to krb5-config
, you need to ensure that the necessary Kerberos development libraries are installed on your system. This is because sparkmagic
uses the requests-kerberos
package, which requires these libraries.
For Debian-based systems (like Ubuntu), you can install the required libraries using the following command:
sudo apt-get install libkrb5-dev
For Red Hat-based systems (like CentOS), use:
sudo yum install krb5-devel
After installing the necessary libraries, you should be able to install sparkmagic
without encountering the krb5-config
error:
pip install sparkmagic
Additionally, the .github/workflows/tests.yml
file for sparkmagic
confirms that libkrb5-dev
is a required system dependency for running tests across multiple Python versions, including 3.11 [1].
Thank you, after install the package it worked!