pkoro / ar-sync

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

argo-egi-connectors is a bundle of connectors/sync components for various data sources established in EGI infrastructure, most notably GOCDB (topology, downtimes), but there's also support for alternative topology grabbing via various VO feeds and weights information via GStat service.

Data is synced in a daily manner and all together with a prefiltered and expanded status messages from argo-egi-consumer represents an input for argo-compute-engine.

Configuration of connectors is centered around two configuration files:

  • global.conf - URLs of sources besides GOCDB and VO, output filenames, avro schemas
  • customer.conf - enlisted jobs for EGI, enlisted all VO'es and their jobs, job attributes and an implicit directory structure where connectors put their data

More info: http://argoeu.github.io/guides/sync/

About


Languages

Language:Python 99.4%Language:Makefile 0.6%