Issue 49227: Add capability to configure maximum default lineage depth, reduce default
labkey-nicka opened this issue · comments
There are a couple of different ways we utilize lineage queries in LKS:
- Via the experiment-lineage.api (this backs things like the lineage views in our apps).
- Via LineageForeignKey which backs Input/* and Output/* columns in queries for experiment related entities.
The (1) way supports a depth parameter which callers can specify. The (2) way, however, does not support a depth parameter and falls back to a default maximum depth. This default maximum depth is 100 for SQL Server and 1,000 for PostgreSQL (see Issue 37332 for the reason these differ). On systems with a lot of data, and specifically a lot of data across a large number of generations, (2) can become a performance bottleneck due to it processing up to 1,000 generations. In general, this amount of information is not useful for day-day usage.
The request here is to either (and/or):
A. Add a configuration option that allows for users to set the default maximum depth.
B. Reduce the default maximum depth on PostgreSQL to a lower value (100?).
Personally, I'm leaning towards both A and B since it will reduce the default but users can opt-in to a higher default. Otherwise, I'd lean towards B before A. For A, I'm thinking a ModuleProperty on the ExperimentModule could work but open to other ideas.