char5280 / ISISFilePolling

A service to poll the ISIS Data Cache, discover new files and push them to a broker for ingestion into other associated projects

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ISISFilePolling

Build Status Coverage Status

Description

A service to poll the ISIS Data Cache, discover new files and push them to a broker for ingestion into other associated projects.

Installation

Requires: Python3.6+, ActiveMQ

  1. git clone https://github.com/autoreduction/ISISFilePolling.git
    1. Optionally setup a python virtualenv to manage package requirements
  2. pip3 install -e ISISFilePolling
  3. pip3 install -r ISISFilePolling/requirements.txt

Configuration

  1. Copy the test_settings.py file to the same directory but rename it to settings.py. You can use the script to do this automatically: python3 ISISFilePolling/src/build_config/setup_test_environment.py
  2. Update the settings.py file to point to desired file systems / services

Running

  1. Setup a cron job to run ISISFilePolling/src/ingest.py

Development

See documentation for developing here: Developer docs

About

A service to poll the ISIS Data Cache, discover new files and push them to a broker for ingestion into other associated projects


Languages

Language:Python 97.8%Language:Makefile 2.1%Language:Shell 0.2%