Integrate with CRIC to fetch list of Datalake endpoints to test
andreaceccanti opened this issue · comments
This JSON document provides a list of active RSEs in the Datalake.
We can fetch the list of endpoints to test starting from the list of RSEs, however we should probably use a different prefix for each endpoint to not mess with RUCIO data.
The fetch-rses-from-cric.sh script provides the list of datalake endpoints in yaml syntax compliant with the variables.yaml
file.
- Updated
variables.yaml
file with CRIC datalake endpoints - Added
endpoint.<endpoint_alias>.path.prefix
key tovariables.yaml
file, which match the RUCIO path (i.e.rses.<endpoint_alias>.protocols[0].prefix
of JSON document) - Added Get ESCAPE path keywork to get correct ESCAPE path from
prefix
- Added warning message in CI if there are new CRIC RSEs (here), filtering by
.rses.<endpoint_alias>.name
- Updated README accordingly
Integrated dynamic fetch of CRIC endpoints in CI (this commit):
- locally (i.e. using
run-testsuite.sh
): run endpoints listed invariables.yaml
- CI (i.e. using
ci/run.sh
): run endpoints listed invariables.yaml
and new CRIC endpoints