Rebuild criticality score project to improve its reliability and scalability.
calebbrown opened this issue · comments
The current implementation of criticality_score available in this repo has been stagnant for a while.
It has some serious problems with how it enumerates projects on GitHub (see #33), and lacks robust support for non-GitHub projects (see #29).
There are problems with the existing signals being collected (see #55, #102) and interest in exploring other signals and approaches (#53, #102 deps.dev, #31, #82, etc).
Additionally, in #102 I propose an approach to improving the quality of the criticality score.
With all this in mind I am planning a rebuild of this project to:
- allow anyone using this project to generate comparable results to someone else
- improve the extensibility of the project to better allow new project sources (i.e. GitLab, et al) and signals (e.g. deps.dev)
- export raw signal data to allow exploration and experimentation (to BigQuery)
- solve GitHub project enumeration problems
I also have a view to normalize GitHub/GitLab/etc API usage with Scorecards to reduce some of the duplication between these projects, which pushes this work towards Go.
The long term view is to automate updates to the raw signals and criticality scores for projects.
I am currently documenting a plan for an initial milestone which I hope to share in this issue soon.
I hope to have parts of the rewrite being pushed to a public branch by the end of April.
Ideally we'd hold off on further PRs until then.