[New Feature]: Store and look up Compressed CSLC files from GRQ ES
philipjyoon opened this issue · comments
Checked for duplicates
Yes - I've already checked
Alternatives considered
Yes - and alternatives don't suffice
Related problems
No response
Describe the feature request
C-CSLCs, in previous design, used a deterministic file name based on the time and burst-id so that they did not need to be indexed in a database. The file name could be computed and the file be directly looked-up from S3.
This has now changed because the SAS requires the filenames to be in a specific scheme of ADT's choosing. This is because SAS parses the filename and reasonable about it even before opening up the file to process it.
Therefore we need to:
- Store and index metadata about the C-CSLCs in GRQ ES. This has already been done by @collinss-jpl. We need to add one more field into this ES index, however: the burst-id and acquisition time cycle index so that these can be looked up in a deterministic way.
- CSLC download job should now populate the the json that it submits into SCIFLO with these C-CSLC file paths, which are stored in S3 LTS.