"values can't be empty (Parameter 'values')" in LowlandModalityDetector
lstefano71 opened this issue · comments
To reproduce the issue, please use this list of values inside MultimodalityDetectionDemo.cs instead of the synthetic one generated in multiple steps by AddRange:
var data = new List<double>(new double[] {
171, 159, 150, 160, 160, 150, 151, 150, 160, 150, 160, 156, 144, 170, 150, 150, 160, 150, 150, 160, 150, 150, 160,
149, 161, 150, 150, 160, 150, 160, 150, 160, 150, 150, 160, 150, 160, 160, 154, 147, 160, 150, 160, 150, 160, 150,
150, 160, 150, 160, 151, 160, 150, 150, 160, 150, 160, 157, 153, 150, 160, 150, 160, 150, 150, 161, 160, 150, 160,
150, 150, 160, 150, 150, 160, 150, 160, 150, 160, 161, 149, 161, 160, 160, 160, 160, 157, 153, 151, 160, 150, 160,
150, 150, 150, 160, 150, 150, 160, 159, 151, 150, 160, 151, 160, 150, 160, 150, 150, 160, 160, 150, 160, 150, 161,
150, 150, 160, 160, 158, 152, 150, 160, 150, 160, 160, 163, 147, 160, 151, 160, 160, 180, 160, 159, 151, 160, 150,
150, 160, 153, 157, 150, 160, 171, 170, 150, 160, 160, 150, 160, 150, 161, 160, 150, 150, 160, 160, 150, 150, 160,
150, 150, 170, 160, 150, 160, 160, 151, 160, 162, 180, 166, 159, 171, 150, 160, 190, 161, 170, 170, 160, 166, 155,
170, 150, 160, 150, 150, 168, 162, 160, 160, 150, 165, 155, 161, 150, 160, 160, 160, 159, 171, 160, 160, 181, 160,
166, 154, 160, 191, 169, 160, 160, 150, 160, 203, 161, 159, 150, 160, 150, 160, 160, 165, 166, 160, 150, 160, 164,
156, 150, 160, 150, 160, 150, 160, 150, 161, 149, 161, 150, 150, 160, 160, 150, 160, 150, 161, 160, 170, 170, 160,
160, 160, 183, 172, 172, 164, 155, 161, 160, 160, 168, 152, 150, 160, 160, 160, 160, 159, 161, 160, 173, 147, 160,
160, 151, 170, 150, 163, 166, 146, 161, 170, 281, 164, 147, 160, 160, 150, 153, 167, 174, 152, 174, 153, 150, 150,
160, 160, 150, 160, 160, 158, 152, 161, 160, 150, 160, 151, 162, 147, 159, 161, 150, 150, 160, 151, 150, 176, 164,
160, 160, 160, 160, 160, 160, 160, 170, 171, 172, 202, 180, 183, 182, 189, 175, 166, 161, 163, 152, 172, 159, 160,
161, 161, 170, 165, 170, 165, 150, 163, 160, 160, 162, 161, 150, 160, 160, 170, 160, 160, 160, 161, 160, 160, 160,
180, 170, 161, 167, 152, 161, 150, 160, 160, 160, 160, 160, 170, 166, 155, 150, 160, 180, 155, 160, 159, 161, 160,
170, 160, 160, 150, 160, 161, 150, 160, 170, 160, 160, 160, 150, 160, 160, 150, 171, 161, 149, 163, 147, 160, 150,
160, 160, 160, 165, 145, 158, 153, 150, 160, 150, 160, 159, 151, 160, 150, 150, 150, 161, 150, 160, 150, 150, 165,
146, 160, 150, 150, 159, 151, 160, 160, 150, 160, 180, 150, 160, 167, 163, 247, 160, 170, 159, 151, 160, 170, 160,
171, 160, 159, 168, 153, 190, 170, 150, 170, 160, 170, 151, 182, 148, 160, 160, 150, 170, 150, 150, 160, 155, 155,
152, 160, 163, 147, 150, 160, 152, 155, 156, 152, 150, 157, 155, 150, 150, 160, 160, 150, 150, 161, 149, 159, 161,
160, 173, 167, 150, 150, 150, 150, 150, 160, 150, 151, 170, 157, 154, 160, 160, 150, 163, 158, 160, 150, 150, 160,
145, 170, 161, 156, 154, 150, 150, 160, 150, 160, 150, 150, 160, 160, 161, 171, 174, 160, 150, 160, 160, 180, 150,
160, 171, 150, 160, 160, 160, 171, 170, 140, 161, 160, 160, 160, 149, 166, 147, 160, 160, 161, 150, 150, 159, 161,
160, 170, 158, 155, 170, 160, 160, 180, 151, 160, 170, 170, 160, 150, 160, 150, 170, 160, 177, 160, 150, 181, 180,
170, 159, 161, 160, 160, 160, 160, 151, 169, 158, 163, 160, 160, 170, 160, 160, 160, 150, 160, 160, 160, 160, 150,
160, 165, 155, 161, 170, 160, 160, 150, 160, 160, 151, 170, 160, 150, 160, 160, 149, 171, 160, 160, 150, 160, 150,
160, 160, 160, 150, 161, 150, 160, 150, 150, 150, 160, 160, 150, 150, 160, 160, 150, 160, 151, 150, 163, 146, 156,
144, 159, 151, 160, 162, 148, 150, 160, 150, 160, 161, 150, 165, 155, 160, 160, 170, 150, 160, 150, 160, 151, 164,
146, 160, 160, 160, 160, 161, 160, 162, 159, 151, 163, 157, 160, 150, 170, 169, 161, 160, 151, 160, 170, 160, 150,
150, 160, 150, 150, 150, 150, 160, 151, 159, 151, 181, 163, 150, 149, 160, 150, 150, 160, 159, 151, 151, 160, 160,
160, 176, 160, 154, 150, 160, 160, 161, 166, 181, 173, 161, 160, 157, 157, 157, 152, 170, 151, 170, 170, 170, 160,
160, 170, 160, 169, 162, 160, 165, 165, 158, 155, 157, 160, 170, 175, 165, 165, 155, 181, 161, 149, 170, 160, 170,
160, 160, 160, 160, 161, 150, 160, 158, 280, 191, 160, 175, 648, 164, 164, 157, 155, 161, 159, 156, 156, 159, 165,
166, 163, 156, 156, 151, 164, 160, 160, 150, 160, 150, 160, 159, 151, 161, 160, 170, 160, 160, 160, 160, 160, 157,
159, 160, 161, 165, 159, 152, 151, 159, 160, 160, 151, 160, 165, 172, 149, 160, 160, 160, 160, 160, 161, 160, 170,
161, 167, 166, 157, 160, 160, 160, 164, 156, 181, 165, 155, 160, 160, 160, 169, 152, 186, 165, 160, 164, 159, 170,
171, 175, 155, 160, 160, 160, 158, 152, 160, 150, 160, 160, 161, 160, 150, 160, 160, 150, 160, 150, 163, 147, 190,
180, 162, 149, 160, 150, 170, 150, 177, 153, 160, 159, 151, 160, 163, 148, 160, 160, 160, 160, 160, 159, 162, 159,
161, 164, 156, 150, 165, 154, 150, 161, 160, 152, 158, 162, 148, 160, 160, 150, 186, 149, 160, 160, 160, 150, 164,
160, 161, 159, 168, 161, 166, 160, 150, 160, 166, 155, 150, 160, 150, 150, 160, 161, 159, 158, 152, 150, 160, 150,
160, 161, 150, 160, 160, 160, 150, 160, 150, 160, 160, 160, 160, 174, 152, 160, 156, 154, 150, 161, 160, 171, 154,
159, 161, 156, 166, 154, 150, 160, 160, 170, 150, 160
});
If you execute the mode detection the program crashes at:
System.ArgumentOutOfRangeException
HResult=0x80131502
Message=values can't be empty (Parameter 'values')
Source=Perfolizer
StackTrace:
at Perfolizer.Common.Assertion.NotNullOrEmpty[T](String name, IReadOnlyList`1 values) in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer\Common\Assertion.cs:line 24
at Perfolizer.Common.Sample..ctor(IReadOnlyList`1 values) in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer\Common\Sample.cs:line 29
at Perfolizer.Mathematics.Multimodality.LowlandModalityDetector.<DetectModes>g__LocalMode|5_3(Double location, Double left, Double right, <>c__DisplayClass5_0& ) in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer\Mathematics\Multimodality\LowlandModalityDetector.cs:line 70
at Perfolizer.Mathematics.Multimodality.LowlandModalityDetector.DetectModes(Sample sample, IDensityHistogramBuilder densityHistogramBuilder, Boolean diagnostics) in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer\Mathematics\Multimodality\LowlandModalityDetector.cs:line 162
at Perfolizer.Mathematics.Multimodality.LowlandModalityDetector.DetectModes(Sample sample) in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer\Mathematics\Multimodality\LowlandModalityDetector.cs:line 27
at Perfolizer.Mathematics.Multimodality.ModalityDetectorExtensions.DetectModes(IModalityDetector modalityDetector, IReadOnlyList`1 values) in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer\Mathematics\Multimodality\ModalityDetectorExtensions.cs:line 16
at Perfolizer.Demo.MultimodalityDetectionDemo.Run() in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer.Demo\MultimodalityDetectionDemo.cs:line 65
at Perfolizer.Demo.Program.<>c.<.cctor>b__3_9() in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer.Demo\Program.cs:line 19
at Perfolizer.Demo.Program.Main(String[] args) in C:\Users\stf\Source\Repos\perfolizer\src\Perfolizer\Perfolizer.Demo\Program.cs:line 47
At the site of the crash LocalMode is called with location = 160, left = 160.00000000000006 and right = 160. Because left > right modeValues is left empty.
The data is real performance data collected with a very granular clock.
This is using commit 435a14d from 9 days ago (the latest as of today).
Cheers.
@lstefano71, thanks for the bug report. Your case exposed a serious problem. When I designed LowlandModalityDetector, I was so focused on continuous distributions that I forgot to consider discrete distributions. Currently, this case is not fully supported. Specifically, if we have a bunch of identical elements in the data, we might get equal values of different quantiles. This prevents us from building a smooth probability density function.
Fortunately, I have a few ideas on how to fix it. Once we detect a "discrete-like" distribution, we could switch to another density estimation approach. I guess we could try classic histograms with a few adjustments. I need some time for experiments, but I hope to support such cases in the nearest future.
Automatic detection of a "discrete-like" distribution would be top-notch, but if you told me to use a different modality detector on this kind of distributions I would not mind: at the moment, in my foreseen use cases, I would always know beforehand if the clock is granular or continuous and would know which detector to use.
By the way: your blogs are eye opening for me. I wish I'd had a teacher like you 30 years ago when I took my master degree in Physics.
Automatic detection of a "discrete-like" distribution would be top-notch, but if you told me to use a different modality detector on this kind of distributions I would not mind: at the moment, in my foreseen use cases, I would always know beforehand if the clock is granular or continuous and would know which detector to use.
Well, right now I can suggest only MValueCalculator.Calculate(data) which returns a double value called mvale. It works great if you want to sort multiple distributions by their modality and find the most multimodal distribution, but it doesn't provide clear information about the number of modes: you have to manually choose a specific threshold for multimodal distributions. You can read more about mvalues here and here.
your blogs are eye opening for me. I wish I'd had a teacher like you 30 years ago when I took my master degree in Physics.
Thanks, I'm glad you like it! Keep tuned, more interesting blog posts are coming.
@lstefano71 the bug is fixed, the fix is available in perfolizer 0.3.0-nightly.93. Currently, it detects 3 modes in your sample (150, 160, 170) which seems to be true:
Note, that the current fix is pretty simple and may still have some issues with discrete-like samples. However, I have a more generic solution: the problem could be solved using jittering. I wrote a few blog posts on this topic:
- Kernel density estimation and discrete values
- How to build a smooth density estimation for a discrete sample using jittering
- Improving quantile-respectful density estimation for discrete distributions using jittering
I'm going to implement this approach in the nearest future.
Thank you very much! I've been following your blog since I've discovered Perfolizer. I did not change my impression: impressive content and very good exposition. I am really happy you are sharing your knowledge with the rest of the world.