SD2E / uploads-manager

(Mirror) Automatically copies files uploaded to designated TACC S3 buckets to their respective Agave-managed storage resource.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Ingest File on Upload

This Reactor responds to upload events by copying the uploaded file from S3 to the destination configured in the settings document. No path remapping is done. The Reactor also makes a couple of attempts to grant READ to world on the file and any directories created as a consequence of the copy. Finally, it messages one or more downstream Reactors with the agave-canonical form of the file path for additional processing.

Example inbound message

{"uri": "s3://uploads/emerald/201809/protein.png"}

Example outbound message

{"uri": "agave://data-sd2e-community/uploads/emerald/201809/protein.png"}

About

(Mirror) Automatically copies files uploaded to designated TACC S3 buckets to their respective Agave-managed storage resource.


Languages

Language:Python 72.9%Language:Shell 23.1%Language:Makefile 3.4%Language:Dockerfile 0.7%