asalt / MSPCRunnerDAG

DAGster pipeline for proteomics

MSPCRunnerDAG

Welcome to your new Dagster repository.

Contents

Name	Description
README.md	A description and guide for this code repository
setup.py	A build script with Python package dependencies for this code repository
workspace.yaml	A file that specifies the location of the user code for Dagit and the Dagster CLI
MSPCRunnerDAG/	A Python directory that contains code for your Dagster repository
MSPCRunnerDAG_tests/	A Python directory that contains tests for MSPCRunnerDAG

Getting up and running

1. Create a new Python environment and activate.

Pyenv

export PYTHON_VERSION=X.Y.Z
pyenv install $PYTHON_VERSION
pyenv virtualenv $PYTHON_VERSION MSPCRunnerDAG
pyenv activate MSPCRunnerDAG

Conda

export PYTHON_VERSION=X.Y.Z
conda create --name MSPCRunnerDAG python=PYTHON_VERSION
conda activate MSPCRunnerDAG

2. Once you have activated your Python environment, install your repository as a Python package. By using the --editable flag, pip will install your repository in "editable mode" so that as you develop, local code changes will automatically apply.

pip install --editable .

Local Development

1. Set the DAGSTER_HOME environment variable. Dagster will store run history in this directory.

mkdir ~/dagster_home
export DAGSTER_HOME=~/dagster_home

2. Start the Dagit process. This will start a Dagit web server that, by default, is served on http://localhost:3000.

dagit

3. (Optional) If you want to enable Dagster Schedules or Sensors for your jobs, start the Dagster Daemon process in a different shell or terminal:

dagster-daemon run

Local Testing

Tests can be found in MSPCRunnerDAG_tests and are run with the following command:

pytest MSPCRunnerDAG_tests

As you create Dagster ops and graphs, add tests in MSPCRunnerDAG_tests/ to check that your code behaves as desired and does not break over time.

[For hints on how to write tests for ops and graphs in Dagster, see our documentation tutorial on Testing.