Make trimmed mean optional (disable option)
alexcasalboni opened this issue · comments
By default, the state machine will discard the top/bottom 20% of "outliers" (the fastest and slowest), to filter out the effects of cold starts that would bias the overall averages.
This behaviour could be optional, with a new discardTopBottom parameter that could take a value between 0 and 0.4, with 0 meaning "no trimmed mean at all" and 0.4 meaning "trim 40% of the top/bottom results" (resulting in only 20% of the results being considered). The default value would still be 0.2, to keep the current behaviour.
Reference: #123
Hello, I opened PR #172 to add the discardTopBottom parameter.
About #123, from I understand how lambda Cloudwatch logs show duration, the cold start is shown in a separate parameter, Init Duration, for example:
REPORT RequestId: d9fab59c-062e-47f3-9570-7e414c43fcc7 Duration: 47.28 ms Billed Duration: 48 ms Memory Size: 10240 MB Max Memory Used: 86 MB Init Duration: 340.74 ms
I made a second PR #173 to extract and calculate the Init Duration for each run. There is an obvious limitation with this time, Init Duration is present only once for each power values, in the first lambda run. Statistically this value is less valid, but I think it could still be useful.