jhermann / ezpy

[ALPHA] :pizza: Make Python application installs easy as pie, for both users and developers.

ezpy

         

Make Python application installs easy as pie, for both users and developers.

Contents

Introduction

What Does it Do?

The main goal of the project is to make installs very easy and convenient, without abandoning security and other technical requirements. Think pipsi with a wider scope, and working on Python 3.

Design Goals

• Reduce installation instructions to ideally one or two command calls.
• Make distribution of ‘simple’ scripts a breeze.
  • … but support dependencies from PyPI nontheless.
• Require only a Python interpreter to be already installed on an end-user machine.
  • … but use what's installed already (if new enough).
  • Work on outdated installations (say down to Jessie / Trusty for now).
• Work on Linux, MacOS, Windows (in that order of importance).
• Rely heavily on existing tooling, especially venv and pip.
• Vendor any dependencies, or don't have any beyond the standard library.

What's Here?

While this repository contains code for a command line tool, it does not do anything yet. At the moment pointing to other projects that fit above design goals proves more useful.

So here is what you get:

• My thoughts on good release and installation work-flows in “Usage”.
• Quick hints on how to use dephell's jail sub-command for easy software installation.
• Deadsnakes PPA builds for Debian in a Docker container, leading to 3.6 … 3.8 being uniformly available on Debian (old-)stable and Ubuntu (old-)LTS, i.e. the four major releases of the Debian-verse.
• A list of related projects, with comments on ones I tried myself, and those I regularly use.

If you're missing something within the range of the stated design goals, open an issue with your user-story, and we'll see if I like it. ☺

Usage

End-Users

Story ♯1: Users can download a simple script and start it (or, well, curl|python it), and it'll bootstrap a full environment of dependencies (dephell jail fits that bill).

Story ♯2: Users need to pip3 install --user ezpy. Calling python3 -m ezpy check after that will go around any OS shenanigans – that could be used to handle the ~/.local/bin problem (python3 -m ezpy fix-path).

Given no dependencies, sudo pip3 install ezpy would also be ok. And this is another option for bootstrapping:

pip install -t /tmp/$USER-$$ ezpy
python /tmp/$USER-$$/ezpy.py bootstrap

Then users are able to install any PyPI application or tool via a simple call to ezpy get|run ‹requirement|script-url›. Mostly like pipsi operates. Console entry points are linked / copied into the existing path, leaving out any unrelated virtualenv binaries (including Python itself).

Story ♯3: ezpy run ‹url›#‹algo›=‹checksum› to directly run scripts from the web – security is based on the trust you have into the 3rd party that gave you the link with the checksum. No difference to PyPI installs secured by a checksum – those run downloaded code, too. dephell jail try … does this already, but I'm unsure about any special security measueres.

Story ♯4: ezpy docker run … – again, dephell has some Docker support, but I did not try that yet.

Developers

ezpy should also be usable to ease the developer workflow. Cloning projects by their PyPI name from the usual sources (GitHub, GitLab, …), as well as cloning a fork of those project made in your account (with both upstream and origin set accordingly).

Then there is support for the end user stories…

Relating to ♯1, give developers the ability to embed code that intercepts command calls (myscript ezpy …). They call ezpy embed ‹myscript.py›, and a stub is added that does this.

For #2, nothing special has to be done. Just release normally to PyPI, or make a script available via a HTTP[S] URL.

♯4: Something like ezpy docker build … or ezpy docker-file ….

Installation

ezpy can be installed via pip install ezpy as usual, see releases for an overview of available versions. To get a bleeding-edge version from source, use these commands:

repo="jhermann/ezpy"
pip install -r "https://raw.githubusercontent.com/$repo/master/requirements.txt"
pip install -UI -e "git+https://github.com/$repo.git#egg=${repo#*/}"

As a developer, to create a working directory for this project, call these commands:

git clone "https://github.com/jhermann/ezpy.git"
cd "ezpy"
command . .env --yes --develop  # add '--virtualenv /usr/bin/virtualenv' for Python2
invoke build check

You might also need to follow some setup procedures to make the necessary basic commands available on Linux, Mac OS X, and Windows.

‘dephell’ Notes

‘dephell’ is a useful add-on tool for project and venv management that works with existing standard tooling, instead of doing a bad replacement job like so many others.

It is installed via a Python installer script into its own venv (compatible to what dephell itself creates as a so-called ‘jail’).

curl -L dephell.org/install | python3  # or 'python3.6'

To get a fully functional ‘git clone’d working directory as of January 2020, this is needed after a ‘normal’ venv setup:

pip install "pip<19"  # fixed in dephell 0.8.1 for pip 20
pip install -e .[full]  # install all optional deps
pip install "mistune<1"  # fix "m2r"

Note that Python 3.6 is needed at minimum.

Deadsnakes PPA Builds for Debian in Docker

See the deadsnakes subfolder for a Dockerfile that builds packages for some Debian releases based on the Deadsnakes PPA code.

For example, Debian Stretch comes with Python 3.5.3 on board, and this allows you to build Python 3.6 packages that can be installed concurrently to the default 3.5 ones.

References

• pip issue #3813
• Share scripts that have dependencies with Nix

Related Projects

Let's start with an overview of tools I used at some point (or which I'm still using) and which proved useful and usable:

• PEX creates single file ZIP apps (PEP 441). shiv is very similar. PEX has some multi-platform support, while shiv is simpler. If one them does not satisfy your needs, try the other.
• platter collects wheels into a tarball for off-line and repeatable deployments. It's sort-of unmaintained. wagon is similar, but I did not test it yet.
• For dephell, see the section above.

What follows is a longer list, in part abandoned / unmaintained, and with different traits regarding platform compatibility and versatility. Last recorded commit activity is included as (YYYY) – at the time entries were initially added or updated. So these dates are an upper bound regarding age, and are not consistent across all entries.

• pantsbuild/pex (2020) – A library and tool for generating .pex (Python EXecutable) files.
• linkedin/shiv (2020) – A command line utility for building fully self-contained Python zipapps as outlined in PEP 441, but with all their dependencies included. [Python 3.6+]
• mitsuhiko/platter (2018) – A useful helper for wheel deployments.
• takluyver/pynsist (2019) – Build Windows installers for Python applications (also cross-platform).
• dephell/dephell (2020) – Python project management. Manage packages: convert between formats, lock, install, resolve, isolate, test, build graph, show outdated, audit. Manage venvs, build package, bump version.
• facebookincubator/xar (2019) – XAR lets you package many files into a single self-contained executable file. This makes it easy to distribute and install.
• getsentry/freight (2020) – A service which aims to make application deployments better.
• armadaplatform/armada (2017) – Complete solution for development, deployment, configuration and discovery of microservices.
• cloudify-cosmo/wagon (2018) – Creates wheel-based archives to allow portable offline installation of Python packages and their dependencies.
• sdispater/poet (2017, experimental) – Declare, manage and install dependencies of Python projects – inspired by cargo.
• jamesabel/osnap (2019) – Deliver self-contained Python applications to end users for Windows and OSX/MacOS.
• glyph/venvdotapp - Virtualenv to NSBundle (MacOS) packager.
• mherrmann/fbs (2018) – Create cross-platform desktop apps in minutes, not months.
• PyAr/fades (2018) – fades is a system that automatically handles the virtualenvs in the cases normally found when writing scripts and simple programs, and even helps to administer big projects.
• flatpak (2017) – Linux application sandboxing and distribution framework.
• jonparrott/noel (2016) – Easily deploy applications to Kubernetes.
• 0xadada/dockdj (2015, unmaintained) – Building 12-factor Python / Django web apps Docker images and deploying them to AWS.
• mattmakai/underwear (2015) – Dead simple LAMP-stack deployments for Python-powered web applications.
• conda/constructor (2020) – A tool for creating installers from conda packages.
• itsjohncs/superzippy (2018) – A simple tool for turning a multi-file, multi-dependency Python script into a single file.