genomon-api

genomon-api is an API server for Genomon pipelines. genomon-api supports the following operations over HTTP:

running DNA/RNA pipelines
tracking analysis states
managing job histories
serving result files

Design overview

genomon-api launches genomon_pipeline_cloud containers through Docker remote API. Samples config .csv and parameters config .cfg are automatically generated from HTTP requests and then mounted to the containers.

Prerequisites

Leiningen >= 2.9.1
Java >= 11
Docker >= 19.03

Developing

Getting genomon_pipeline_cloud

You need to get a docker image of genomon_pipeline_cloud. If you don't have it, build one with the following command:

git clone https://github.com/chrovis/genomon_pipeline_cloud.git \
  && cd genomon_pipeline_cloud \
  && docker build . -t genomon_pipeline_cloud:latest

You can use other images by setting environment variables GENOMON_DOCKER_IMAGE and GENOMON_DOCKER_IMAGE_TAG.

Setup

When you first clone this repository, run:

lein duct setup

This will create files for local configuration, and prep your system for the project.

To launch the system, you need to supply configs for AWS. docker-compose will read values from environment variables and .env file.

cp .env.example .env
vim .env
docker-compose up --build
# access http://localhost:8000 to open Swagger UI

You need to export .env file to access these values from lein repl.

eval "$(cat .env | sed -e 's/^/export /g')"
lein repl

You can use a local config file (dev/resources/local.edn) for changing configs without restarting JVM. For example:

{:genomon-api.aws/profile-credentials-provider
 {:profile-name "..."},

 :genomon-api.aws/client
 {:credentials-provider #ig/ref :genomon-api.aws/profile-credentials-provider},

 :genomon-api.executor.genomon-pipeline-cloud.config/instance-option
 {:aws-subnet-id "...",
  :aws-security-group-id "...",
  :aws-key-name "...",
  :aws-ecs-instance-role-name "..."}

 :genomon-api.executor.genomon-pipeline-cloud/executor
 {:env {"AWS_ACCESS_KEY_ID" "...",
        "AWS_SECRET_ACCESS_KEY" "...",
        "AWS_DEFAULT_REGION" "..."},
  :output-bucket "s3://..."}}

For details of required privileges, please refer to the setup documents of ecsub and genomon_pipeline_cloud.

Environment

To begin developing, start with a REPL.

lein repl

Then load the development environment.

user=> (dev)
:loaded

Run go to prep and initiate the system.

dev=> (go)
:duct.server.http.jetty/starting-server {:port 3000}
:initiated

By default this creates a web server at http://localhost:3000.

When you make changes to your source files, use reset to reload any modified files and reset the server.

dev=> (reset)
:reloading (...)
:resumed

Testing

Testing is fastest through the REPL, as you avoid environment startup time.

dev=> (test)
...

But you can also run tests through Leiningen.

lein eftest

Build

Jar

To build a standalone jar, run:

lein uberjar

Docker

To build a docker image, run:

docker build . -t genomon/genomon-api:latest

chrovis-genomon / genomon-api