New metric under Growth-Maturity-Decline
rpaik opened this issue · comments
All,
Before I do another pull request, I thought I discuss this to get a consensus. Under the "code development" section of G-M-D, I'm thinking of another metrics for number of code reviews over a time period. In OPNFV Gerrit, I wanted to know for each contributor how many reviews they have done (and even how many (sub)projects they contributed reviews to) over a time period and realized that this was not part of our dashboard. After talking to Dani at Bitergia, I created a feature request in the Grimoirelab repo at chaoss/grimoirelab#58
Any thoughts/feedback on this?
I do think this is a relevant metric, since it captures a part of the activity on reviewing. This could be complemented with how many of them were positive and how many negative, to have an idea about the performance of the process (ideally, it should be "everything positive", which would mean that proposed patches are of outstanding quality, and never deserves a negative review, of course assuming reviewers are really reviewing).
(Yes, we have this in our todo list for GrimoireLab...)
Agreed. I think this is a nice metric and would also be a nice metric relative to others as has been talked about in weekly meetings.
Reviews and relation to Commits
Reviews and relation to PRs
To clarify, the metric should be on a 'per person' basis?
Will it be anonymous or exposing people's identities?
@GeorgLink, that's actually a good question. If you look at our current metrics for commits or number of lines changed, it's anonymous. Should we keep everything anonymous or change the metrics to say e.g. "number of commits per contributor?"
@rpaik I think the metrics should definitely stay agnostic to the item being tracked (i.e. how it currently is). If you start designating by things like "per contributor" you open up a can of worms for comparable items such as per organization / company / gender / etc. These designations are possible for the majority of the metrics we have, and it would be too much to separate these for each one.
As for a metric for code reviews, I agree 100%, this is something I already track a little bit of via git repos, and it's definitely a useful metric. We probably just need one simple metric under Code Development. Something like: Code Reviews | Total number of code reviews
This is relevant to a discussion I´m having with @sgoggins on how to structure metrics. The problem is that in many cases we want to capture:
- how we filter the data (how we select the data relevant), such as "during the last year", "for all contributors active during the last three months", or "for contributors for the 5 most active companies"
- how we partition the data (how we bucket it), such as "per month" or "per contributor",
- how we aggregate the data in each partition (bucket), such as "total count", "mean", or "unique count"
And after that, we even want to visualize the data in different ways (but that's another story).
In Kibana/Elasticsearch, those three are different gadgets: filters, buckets and aggregations are their names. But the same concept is found everywhere: in SQL, for example, it is "FOR", "GROUP BY", and "SELECT" (well, sort of).
So, trying to express it in these terms, I think we usually refer to the metric as the "aggregation". In our case here, the "unique number of code reviewers". That could be filtered for a certain period of time (like the last year), and bucketed by month (to track its evolution over time). Or bucketed by organization, to learn about the organizations contributed with more reviewers. Of by gender, to learn about diversity in code review.
@BenLloydPearson I agree with your points that it'd be good to stay consistent with what we have today. I'll go ahead and make a pull request....
Thanks everyone!
I like consistency. However, if you are more interested in knowing something different, then we should also consider a way to incorporate it in our metrics work. We can always include a more obscure metric in the activity-metrics-list
Review-Commit Ratio | Ratio of Reviews/Commits for the top 5% of committers - indicates a culture of balancing creative and maintenance work
What is the state of this issue? Can we close this?
#50 was the pull request that came out of this conversation and added "Number of Reviews".
@rpaik had originally expressed interest in the question, how much individual contributors review and on which projects. As @jgbarah pointed out, by tracking number of reviews, the filter can be adjusted to show the reviews by individual contributors and projects.
-- Issue resolved!