s3ql / s3ql

a full featured file system for online data storage

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Provide better instructions for checking if dependencies are installed

automaticit-anthonyyen1 opened this issue · comments

First of all, thank you and every contributor for the project.

In S3QL 5.2.0 documentation » Installation » Dependencies, the section says:

To check if a specific module is installed, execute python3 -c 'import ; print(.version)'. This will result in an ModuleNotFoundError if the module is not installed, and will print the installed version if the module is installed.

Following these instructions, we found out modules apsw, systemd, google-auth, and google-auth-oauthlib do not support the __version__ attribute, and thus throw errors with the above code. The more Pythonic way of checking PyPI-sourced modules is discussed in the StackOverflow thread Adding version attribute to Python module - Stack Overflow.

Because python3-systemd is not sourced from PyPI, we need to validate it differently, so we will leave it out of the following code that checks the PyPY-sourced modules. The Linux Bash shell code to perform the validation would then be:

for python_module in setuptools			\
		     cryptography		\
		     defusedxml			\
		     apsw			\
		     trio			\
		     pyfuse3			\
		     pytest			\
		     requests			\
		     google.auth		\
		     google_auth_oauthlib	\
		     pytest_trio		\
		     ; do
  echo -n ${python_module}" "
  python3 -c 'from importlib.metadata import version; print( version("'${python_module}'") );'
done

Sample output looks like:

setuptools 67.7.2
cryptography 41.0.7
defusedxml 0.7.1
apsw 3.42.0.1
trio 0.24.0
pyfuse3 3.3.0
pytest 7.3.2
requests 2.31.0
google.auth 2.29.0
google_auth_oauthlib 0.8.0
pytest_trio 0.8.0

The systemd module from systemd/python-systemd: Python wrappers for systemd functionality says, contrary to the Dependencies section, that there is a PyPI-sourceable module version of it, but we haven't tried it yet with S3QL because we're still trying to build S3QL so we're strictly following the S3QL documentation:

The project is also available on pypi as systemd-python:

Therefore, checking for the Python systemd module for S3QL involves an distribution-specific look at the packages installed from a repo or manually identifying the version compiled. On Fedora, the code to check for the version of the module when using the module's documentation to install via dnf would be:

rpm_package_python3_systemd_query_output="$( dnf list python3-systemd 2>/dev/null | sed -e '1,/Installed Packages/d' -e '/Available Packages/,$d' )"
rpm_package_python3_systemd_exists=$( echo ${rpm_package_python3_systemd_query_output} | grep python3-systemd | wc -l )
if [ "${rpm_package_python3_systemd_exists}" -eq 1 ] ; then
  echo "PASS: RPM package python3-systemd exists (details: ${rpm_package_python3_systemd_query_output})"
  prerequisites_exists_psmisc=true
else
  echo "FAIL: RPM package python3-systemd not found"
  prerequisites_exists_psmisc=false
fi

We're sorry if a way to create Pull Requests for documentation was described somewhere and we missed it, but we propose the Dependencies section be modified to the following wording.

To check if the required PyPI-sourced Python modules (all except systemd) are installed and their version level, run the following shell code. This will print the installed version if the module is installed.

for python_module in setuptools			\
		     cryptography		\
		     defusedxml			\
		     apsw			\
		     trio			\
		     pyfuse3			\
		     pytest			\
		     requests			\
		     google.auth		\
		     google_auth_oauthlib	\
		     pytest_trio		\
		     ; do
  echo -n ${python_module}" "
  python3 -c 'from importlib.metadata import version; print( version("'${python_module}'") );'
done

Sample output looks like:

setuptools 67.7.2
cryptography 41.0.7
defusedxml 0.7.1
apsw 3.42.0.1
trio 0.24.0
pyfuse3 3.3.0
pytest 7.3.2
requests 2.31.0
google.auth 2.29.0
google_auth_oauthlib 0.8.0
pytest_trio 0.8.0

If this documentation change is accepted, then we'll investigate whether the PyPI-sourced systemd module can be used with S3QL and report back in a separate issue to improve the instructions, once we work through the S3QL build issues on our Fedora system. Thanks again.

Thanks for the suggestion! I'd rather not have the the 10+ code fragment in the documentation as-is, because it largely just repeats the previous bullet points and would need to be kept in sync by hand.

That said, but I'd be happy to merge a patch that corrects the example, i.e. replace

To check if a specific module is installed, execute python3 -c 'import <module>; print(<module>.__version__)'.

with

To check if a specific module is installed, execute `python3 -c 'from importlib.metadata import version; print( version("'${python_module}'") );'

What do you think?