Calculating velocity metrics differently

[Daniel Skryseth]

Hi, I am wondering if commit sizes should have a more significant role and be included in the calculations?

As I understand, the deployment frequency is calculated by "how often an organization successfully releases to production" [1] as their successful deployments. However, as far as I know, this does not seem to include the features (e.g., deletions and additions) of the code that the successful deployment constitute. Wouldn't it be more appropriate to calculate a deployed value before the deployment frequency, in order to ensure better construct validity? I.e., summarize successful deployments as the total sum of the product of the code included in the deployments. Ergo, deployed value can be calculated as:

$$ f(\textrm{additions}, \textrm{deletions}) = \sum_{i=1}^{n}(\textrm{addition}{i}+\textrm{deletion}{i})$$

Before being classified by the Four Keys tiers as the deployment frequency (e.g., Daily bucket). Where $f$ returns the deployed value, given a list of code features (in this case additions and deletions), for the successful deployments. Not that this would necessarily have to be limited to code additions and deletions (although this metric is easy to collect with such as Git/Github), but could also contain other measures such as files changed. Thus the construct validity may have been improved because it is possible to differentiate between smaller and bigger deployments. Rather than just measuring the successful deployments in themselves. E.g., it is likely that a larger successful deployment takes more time to push than a smaller successful deployment.

This may also apply to lead time for changes, which tries to measure the time it takes to go from "code committed to code successfully running in production" [1]. Considering that this too seem to not factor in the size of the code that is being committed? One way to include this could be to calculate the lead time per change before calculating the overall lead time for changes. I.e., divide the lead time for commited code on its own code features (e.g., additions and deletions):

$$ t(\textrm{additions}, \textrm{deletions})=\sum_{i=1}^{n}\frac{d(x_{i}, y_{i})} {\textrm{addition}{i}+\textrm{deletion}{i}} $$

Where $d(x_{i}, y_{i})$ would be the distance function that takes the difference of two timestamps (e.g., creation and merging dates) $|x-y|$ and converts it into seconds. This time is then divided on the code features and ultimately summarized and returned by the lead time per change $t$ function. For example, given creation date Friday May 15, 2020 13:39 UTC and merge date Friday May 15, 2020 20:32 UTC for a PR, where the size of the code features (additions and deletions) from the commits are equal to 5.5. Then the lead time per change would be 4,505, meaning it took 4,505 seconds to make the change for the PR. The same logic may then also apply more abstractly to "when the commit happened, and when the deployment happened" [1].

Lead time per change will then be able to factor in that it could take a shorter time to make changes to smaller commits, and vice versa. Rather than just summarizing the lead time for the time between when a commit happened and the deployment happened. Thus potentially increasing the construct validity further, as it may lead to a more reliable lead time for changes measure.

[1] - https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance