Improve the package-ability of hydroffice.bag
ghisvail opened this issue · comments
Following our discussion started on the hdf-compass issue tracker, I made an attempt to package hydroffice.bag for Debian but hit a few issues. The package overall builds fine with the current Python 2 and Python 3 stack on Debian, but its final content does not meet the standard of our packaging policies.
The main problem concerns the package_data, which are installed under the Python system site-packages path (/usr/lib/pythonX.Y/dist-packages/...) instead of the recommended architecture independent path (/usr/share/...). The code in the helper module does not apparently handle arbitrary paths (only paths relative to the module file) so just patching setup.py to install the files listed in package_data somewhere else won't work.
On my end, the goal is to achieve a binary packaging organized the following way:
- a Python 2 package to use the
hydroffice.baglibrary with the Python 2 interpreter - a Python 3 package to use the
hydroffice.baglibrary with the Python 3 interpreter - a "common" package containing the
package_datafiles shared between the Python 2 and 3 packages - a "doc" package containing the HTML documentation
- a "tools" package containing the command-line scripts
Thanks for your help.
@ghisvail Thank you for the suggestions! Do you have some hints to existing compliant packages that have similar requirements (a package_data) so that I can follow their approach?
I will work on them during next week (together with some other planned improvements). I will contact you as soon as I have a 2.7 RC to test if it is compliant with the Debian policy.
@ghisvail I looked around to find Python packages with such an architecture to see how to modify the setup.py.. but I didn't have luck. I understand the advantages of the structure that you describe.. but I am not sure how to properly implement it.
For instance, this is currently a pure Python library that works with both Python 2 and 3 without six. So what do you mean with a Python 2 package [for] the Python 2 interpreter, and a Python 3 package [for] the Python 3 interpreter?
BTW, I am also attempting to follow: https://python-packaging-user-guide.readthedocs.org/en/latest/
I am not sure how to properly implement it.
I can work around it with symlinks. Please disregard this specific comment.
So what do you mean with a Python 2 package [for] the Python 2 interpreter, and a Python 3 package [for] the Python 3 interpreter?
Means one package installs h.b in the system Python 2 dist-packages, and the other package in the system Python 3 site-packages.
BTW, I am also attempting to follow: https://python-packaging-user-guide.readthedocs.org/en/latest/
These are good practices indeed.
Can I rely on this repository to sync the packaging with, or shall I use the tarballs on PyPI instead?
Pypi wheels and github tag releases should be in sync. Drop me a line if you discover any issue on that.
setup_args['install_requires'] =\
[
"lxml",
"numpy",
"gdal<2.0",
]
Debian has gdal >= 2.0. Any plans to support this version?
The issue with gdal>=2.0 is strictly related to the freezing:
As gdal 2.1.0 that fixes the issue has been released a few hours ago: https://pypi.python.org/pypi/GDAL/2.1.0 , the question really would be: any plan for Debian to support gdal 2.1.0 ?
The issue with gdal>=2.0 is strictly related to the freezing
Since I don't use pyinstaller, it should be fine then?
any plan for Debian to support gdal 2.1.0 ?
The Debian GIS team is quite active, so I expect gdal 2.1.0 to come really soon.
New tag without the strict GDAL requirement: https://github.com/hydroffice/hyo_bag/releases/tag/v0.2.12
Thanks
Ok, now I have a candidate package based on v0.2.12.
Great!
What is the purpose of the media/* files listed in package_data?
To install the library icons
Are you planning to provide source tarballs on PyPI alongside the wheels in the future? Or should I rely on Github's ones?
One big advantage (for us) in providing a tarball on PyPI is that the source can be curated from the useless (for us) CI scripts and VCS files.
I think that the PyPi solution should be always preferred: https://pypi.python.org/pypi/hydroffice.bag
Let me know if the tarball meets your requirements
Well, the docs and tests are missing, which means neither a documentation package can be built nor any testing can be performed. For the rest (examples and specs), it depends whether you think they would be worth providing to the end-user, perhaps in a separate package. Your call.
I have added docs, tests, and examples. I don't see the point to add specs. Give it a look..
Looks good to me, I'll report if I am missing anything else. Packaging coming soon.
The tarball is missing AUTHORS.rst and HISTORY.rst required by setup.py to build the long description.
Corrected. Thank you!
Ok the last warnings I get from the packaging come from the absence of manpage for the console scripts. I'll attempt a submission without those, since I don't feel qualified to write them. However, it would be nice to think about generating these manpages later.
There are multiple options for solving this:
- Use help2man in a custom build command. (for instance in obspy)
- Write them from scratch with sphinx.
- Use a sphinx plugin such as autoprogram.
The last option is probably interesting for you, since all your scripts use a parser from argparse.
Let me know if the submission is rejected for the missing manpages. However, we will most likely adopted in a short time the autoprogram solution given its simplicity.
hydroffice.bag is now available in Debian.
Great!