metacpan / metacpan-developer

Deprecated developer env. Please use https://github.com/metacpan/metacpan-docker

!! Deprecated !!

Please do not begin any new development using this project

All new development should be done using https://github.com/metacpan/metacpan-docker

!! Deprecated !!

MetaCPAN Developer

• Initial Setup
  • Requirements
  • Setup Repos and VM
  • API and Web Interface
  • Fork the Repo
• Workflow
  • Run the Tests
  • Make a Change
  • Restart the Service
  • Update the VM
  • Make a Pull Request
• Creating a New Box From Scratch
• Getting Help

This is a virtual machine for the use of MetaCPAN contributors. We do not recommend installing manually, but if you want to try it there are some instructions in the puppet repository.

For information on using MetaCPAN, see the api docs.

Initial Setup

Requirements

• Vagrant (1.2.2 or later)
• VirtualBox, we recommend 4.3.10, see guest additions if you get mounting issues
• A git client
• An ssh client if not built in, Windows users see this.
• To be able to download about 900MB of data on the first run

Set Up Repos and VM

git clone git://github.com/metacpan/metacpan-developer.git
cd metacpan-developer
./bin/host/init.sh # recursively clone all of the metacpan repos

If you have an older box (pre Debian stretch): vagrant destroy so that you can build a new box.

VAGRANT_VAGRANTFILE=box-builder/Vagrantfile vagrant up
VAGRANT_VAGRANTFILE=box-builder/Vagrantfile vagrant halt
DEBIAN_FRONTEND=noninteractive vagrant up --provision

This will run an initial provision with the box-builder Vagrantfile. Once the initial bootstrapping is in place, we just re-provision with the standard Vagrantfile.

vagrant provision can be run multiple times, and includes running the puppet setup (which will also install any Carton dependencies, though there are instructions below for doing this manually as well). Warnings in the puppet setup (in red) are usually ok, Errors are not.

At this point you have a virtual machine with all of the MetaCPAN services up and running! You can connect to it with:

vagrant ssh

Seed Elasticsearch

Running this script will set up the Elasticsearch mappings, fetch some CPAN modules and index them for you.

vagrant ssh
bash /vagrant/bin/guest/index-cpan.sh

API and Web Interface

The API and web interface are also forwarded to ports on your machine: 5000 and 5001 respectively.

For simplicity, from now on we'll assume you're working on metacpan-web. Working on the API is very similar - you can essentially replace 'web' with 'api' in the following instructions - except that you need to get some test data to use. Instructions for that are here.

Note that the web service connects to the actual metacpan api, not your local one. If your patch changes both and you need to test them together, instructions for connecting them are in the FAQ.

Fork the Repo

You'll need to make a fork of the repository you're working on; then instead of cloning that to your local machine, you can point the copy you already have to it.

cd /path/to/metacpan-web
git remote rename origin upstream
git remote add origin [e-mail]:username/metacpan-web.git

Workflow

Running plackup in Your Terminal

./bin/run plackup --port 5001

metacpan-web runs on port 5001. metacpan-api runs on port 5000.

Run the Tests

You'll want to run the suite at least once before getting started to make sure the VM has a clean bill of health.

NOTE: ./bin/prove is not the system prove but the one in the metacpan-web (or metacpan-api) bin directory.

 vagrant ssh
 cd metacpan-web
 ./bin/prove t

This recursively runs all the tests in the t directory in metacpan-web. To do a partial run during development, specify the path to the file or directory containing the tests you want to run, for example:

./bin/prove t/model/release.t

This will save time during development, but of course you should always run the full test suite before submitting a pull request.

Make a Change

The init script you ran during the initial setup cloned all of the metacpan repositories; then the Vagrant provisioning script mounted those repositories on the VM.

So you can open a separate terminal and code on your own machine:

cd /path/to/metacpan-developer/src/metacpan-web
git checkout -b MyBranch
# hack hack hack
git add somefile
git commit -m"comments in somefile/ now are fully compliant/ with haiku spec, yay!"

The changes you make will show up on the VM and can be used next time you run the test suite.

Installing Dependencies

If you need to add or update a module, make the change in the cpanfile of the repository. In order to install the module you can either run vagrant provision (which is slow) or run a carton wrapper for your repository (which is fast). For example, if you've added a dependency to metacpan-api/cpanfile, you can run sh /home/vagrant/bin/metacpan-api-carton install.

Restart the Service

The projects use Carton to manage dependencies. This is very useful if you are, or might later be, working on more than one of them. Even if you choose to install modules globally on the VM, remember to add them to the repository's cpanfile.

If you've updated the cpanfile, you can run metacpan-web's carton from the vagrant user's home directory:

cd ~
./bin/metacpan-web-carton

Next, restart the web service:

sudo service starman_metacpan-web restart

Now you can go back and run the tests as described above. Repeat until your code is sufficiently awesome.

Update the VM

If you're working on a long-lived branch, you should update the VM periodically. Pull the master branch of metacpan-developer and all of the repos in src:

cd /path/to/metacpan-developer
git checkout master
git pull
cd src/metacpan-puppet
git checkout master
git pull
# etc

Then shut down the VM and bring it up again to ensure you are running the correct versions of all dependencies:

vagrant halt
vagrant up --provision

You may wish to write a script to automate this process.

Make a Pull Request

You'll need to rebase off of master, which can be done on your machine:

cd /path/to/metacpan-web
git checkout master
git pull
git checkout MyBranch
git rebase master

Resolve any conflicts, then restart the service and run the test suite.

Here's a simple PR checklist:

• I've run the entire test suite and it passes
• I've committed all of the changes that are necessary for my patch to work
• I've added tests to show how my patch is supposed to behave
• I've added or edited comments and documentation as appropriate

Now you can push to your fork and create a pull request - we look forward to seeing your work!

Getting Help

First, check the FAQs page for common issues faced during the development process. If your problem isn't solved there, join us on #metacpan (irc.perl.org), or open an issue.