NumPy Vendor

This builds Linux and Windows release binaries and sources for NumPy using a Vagrant VM. It should be 100% reproducible. You need Vagrant and Fabric.

The NumPy release notes are here:

https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt

But most of it is automated by numpy-vendor, so once the release branch is approved for release, just follow the instructions below. Note that Fedora requires root to manage virtual machines.

Fedora 22 prep

numpy-vendor will not run out of the box on Fedora 22. In order to use kvm/qemu for the virtualization the following need to be installed. Some of these may be dependencies of the others.

fabric
vagrant
vagrant-libvirt
libvirt
libvirt-devel
libvirt-daemon-kvm
qemu-kvm

The box file needs to be libvirt compatible. That can be achieved with the following commands.

$ vagrant plugin install vagrant-mutate $ vagrant box add precise32 http://files.vagrantup.com/precise32.box $ vagrant mutate precise32 libvirt

The firewall needs to be opened up for nfs.

$ sudo firewall-config

The default zone should be WorkStation (or some such). For that zone allow

nfs
rpc-bind
mountd

The default udp port range should be OK, needs to include 2049.

Create an .ssh/config file to avoid many warnings:

$ touch /home/charris/.ssh/config
$ chmod 644 ~/.ssh/config

How To Use

Do:

vagrant up
fab vagrant prepare

Log in and manually inspect the numpy repository, the release branch will be checked out. Do some last minute fixes, checkout a different branch, etc.:

vagrant ssh
cd repos/numpy
# Do any changes that are not in official repositories

First, change/check the following variables in pavement.py depending on the release version:

RELEASE_NOTES = 'doc/release/1.7.0-notes.rst'
LOG_START = 'v1.6.0'
LOG_END = 'maintenance/1.7.x'

Do any other changes. When you are ready to release, do the following changes:

diff --git a/setup.py b/setup.py
index b1f53e3..8b36dbe 100755
--- a/setup.py
+++ b/setup.py
@@ -57,7 +57,7 @@ PLATFORMS           = ["Windows", "Linux", "Solaris", "Mac OS-
 MAJOR               = 1
 MINOR               = 7
 MICRO               = 0
-ISRELEASED          = False
+ISRELEASED          = True
 VERSION             = '%d.%d.%drc1' % (MAJOR, MINOR, MICRO)

 # Return the git revision as a string

And make sure the VERSION variable is set properly. Commit it:

git commit -a -m "REL: Release 1.7.0rc1"

Push this commit into your local github account, not the official repository yet, for example by:

vagrant ssh
cd repos/numpy
git push git@github.com:certik/numpy maintenance/1.7.x:release_test

which will work, because vagrant ssh forwards the ssh keys.

Do the release (build general and windows binaries from the repos/numpy directory in the Vagrant VM):

fab vagrant numpy_release

And the directory release will be created in the current directory from the VM. If you need anything else, just login using vagrant ssh and copy it to /vagrant inside the VM.

If everything worked correctly, get the release commit from your github, tag it:

git tag v1.7.0rc1

and push this commit and tag into the official repository. Finally, upload the binaries.

Expert Usage

If you want to start from scratch, but want to reuse the existing vagrant image, you can keep all the deb packages, but reinstall everything else (NumPy, Wine, ...) from scratch. Do:

fab vagrant remove_userspace
fab vagrant prepare_userspace

This will delete all builds and other possible changes that you made into the vagrant image. This is equivalent to fab vagrant prepare, but faster.

Mac Binaries

First prepare the Mac build box as follows:

Install Python 2.7, 3.3 and 3.4 from python.org using the dmg disk image
Install setuptools and bdist_mpkg into all these Pythons
Install Paver into the default Python

Tip: Add the /Library/Frameworks/Python.framework directory into git and commit after each installation of any package or Python. That way you can easily remove temporary installations.

After Linux and Windows release files are in the release directory, we generate Mac binaries. First we setup the environment:

fab -H mac_host mac_setup

where mac_host is the Mac OS X computer which will be used to build the binaries. The only prerequisite is that the release directory contains the reference.pdf and userguide.pdf files generated by the Vagrant VM in the previous step. You can login manually and do some changes in the numpy directory if needed.

Then generate the binaries by:

fab -H mac_host mac_numpy_release

The Mac binaries will be copied to the release directory locally. To remove temporary files from the Mac, use:

fab -H mac_host mac_remove_userspace

Append the checksums at the end of README.txt in the release directory:

md5sum *.dmg >> README.txt

SourceForge

Upload all the files from the release directory to SourceForge:

http://sourceforge.net/projects/numpy/files/NumPy/

Lookup the last release to see which files need to be uploaded and double check that there are no files missing.

rgommers / numpy-vendor