OPERA-SDS-VnV-18: Ingest Static Ancillary Data
LucaCinquini opened this issue · comments
Requirement: "Demonstrate the that PCM system can successfully ingest ancillary files "offline", and use these files during operations."
Things to do:
o Verify that static data for DEMs and Land Cover is stored on S3 buckets
o Submit a PGE via standard procedure
o Verify that the PGE uses static data by looking at the RunConfig and job log file
@LucaCinquini - curious, what does offline mean exactly in this context?
I think it means "one-off" - i.e. data is copied to some S3 bucket and retrieved from there, without dynamic queries to some external service.
Work on the DEMs for now, leave the Land Cover for later.
Please restructure the ticket as follows:
o Verify that the 3 types of ancillary files are stored on S3 bucket (DEMs, Land Cover, World Cover)
o Execute a PGE, and look at the logs to verify that the PGE has used the 3 ancillary data types (DEMs, LC, WC)
o Completely remove section 6
New approach:
o Use the AWS console to navigate to the S3 buckets where the 3 categories of ancillary files are stored: DEMs, Land Cover, and World Cover. Verify that these files are complete (by checking the expected total size) and of the desired version (how?)
Question: are these buckets different for DEV, I&T and Ops? Or are the buckets shared across venues?
o Use the on-demand UI to trigger one (or more) jobs. Wait for the PGE job to start and complete. Look at the job log to verify that the PGE has extracted the 3 categories of ancillary files for the geographic region of interest
(Question: ask PGE team if this information is in-fact stored in the log)
See new version at: https://cae-testrail.jpl.nasa.gov/testrail/index.php?/cases/view/1127058