elastic indexes could be duplicated with a prefix
caseyjlaw opened this issue · comments
The elasticsearch indices could support more diverse data (e.g., aws index, testing index) if the realfast elastic module allowed prefixes. So instead of defining an index as "cands", it can define a prefix as "aws", which is internally setting the index to "awscands". Note that all indices need to be a plural, so that the doc_type is correctly inferred (e.g., as "awscand").
One path forward is to use a prefix of "" so all indices are "cands", "scans", "preferences", "mocks" as the default.
The portal front end will need to be updated to use this, too.
Implemented.
Now using a set of indexes "new", "final", "test", "aws". The first is prefix defines a set of indices that are a landing point of candidate detections. A manual process (in realfast.elastic module) allows migration to the "final" indices.