Feature: discard worst frames first when averaging
theChaosCoder opened this issue · comments
Idk if you know this plugin https://github.com/dubhater/vapoursynth-median/
It does more or less the same as yours, but it has a cool feature where you can discard the worst frames and averages only with the better quality frames.
I used it on some DVD openings / endings. it worked very good, all blocks where gone :)
low
Number of the lowest values to discard after sorting.
Default: 1.
high
Number of the highest values to discard after sorting.
Default: 1.
p.s. could you zip your releases? I could then add it to vsrepo https://github.com/vapoursynth/vsrepo
I did not actually know of that plugin, but then again I should've guessed that someone would've written it for avisynth, and that dubhater would've ported it. The High and Low parameters were actually something I was planning on adding to average.Mean, but then I was worried about the slowdown this would cause due to having to sort the pixels. I'll look into an efficient way of doing this.
Also, I've zipped v0.3.0, and will continue to do so from now on. Feel free to add to vsrepo 👍
hmm v0.3 is 32bit not 64?
Wait is it really. Hang on a moment.
Edit: Okay yeah I did, turns out my laptop is 32 bit. Fun.
Okay, the above has been fixed.
I'm working on this in the pepega branch (named as such because I wrote the code quickly last night, and it's very bad). Just a small question, would you mind it being a default "discard" parameter instead of manually specifying high and low? It's very slightly easier to implement that way, and I don't see a usecase where you might want to discard the top x values but not the bottom x values. (i.e. you'd just end up darkening the frame overall).
If you can think of a good reason to leave high and low separate, then do let me know so that I can implement that instead/
I thought high/low means relative to median, for example here -2 -1 0 1 2 it would discard -2 and 2. But now I'm not sure if I understood high/low correctly...
I'd have to look at the source a little more, but I don't know that doing that for median would make any sense, as in both cases the "median," or middle number, would stay the same anyway. The documentation is as follows:
MedianBlend can return a clip derived from the minimum or maximum pixel values, or it can discard some low (default: 1) and high (default: 1) values and blend the others together. With the parameters set to not discard anything, the result is in fact the average of the clips.
Even if this isn't technically correct, I still think it's a useful feature, as it allows you to get rid of values that are too far from the rest of the group, increasing the accuracy of the mean overall.
I could test both approaches with my 11 (or so) opening clips from a DVD. I'm also curious how much impact he multipliers parameter would have. P/B frames in mpeg2 can look very blocky in fast moving scenes.
That was the main reason behind implementing per frame multipliers. Anyway, I've implemented
a discard parameter in pepega which needs a little cleanup, but it does work. It makes very little difference to the BDs ive been testing with, but I think it would be far more interesting with DVDs.
Here's a copy of my build with discard if you wanted to try it out for now. Be aware that it doesn't have any real safeties if you put in bad parameters yet, so if you aren't careful it'll just crash
vs_average.zip. Just make sure to use a discard value less than half the number of clips you're inputting, and don't specify both discard and preset at once.
Edit: I'll probably end up fixing this enough to be in a test release later today, since the main issue is syntax handling, though there are some memory based optimisations that need to be done before I'm happy to open a PR. (i.e. reallocating vector for every pixel when it could reuse the same vector)
I just tried the test version but it seems the discard parameter is not present.. vapoursynth.Error: Median: Function does not take argument(s) named discard (also tried Mean())
clips = [c4, c5, c6, c7, c8, c9, c10, c11, c12]
clips = [core.fmtc.bitdepth(clip, bits=16) for clip in clips]
clip = core.average.Median(clips, discard=3)
EDIT
vapoursynth.Error: Mean: Function does not take argument(s) named preset
I think there are some parameters missing ^^"
def Mean(self, clips: typing.Union[typing.Sequence["VideoNode"], None] = None, output_depth: typing.Union[int, None] = None) -> "VideoNode"
def Median(self, clips: typing.Union[typing.Sequence["VideoNode"], None] = None) -> "VideoNode"
Maybe I accidentally uploaded the wrong build? Anyway, I know the latest pre-release build (v0.4.0-beta) definately works now. Median doesn't have a discard parameter, since if you discard the top and bottom values of a list, and then take the median, you'll still get the same thing anyway. Only Mean has added functionaility.
The current average.Mean works as follows,
core.average.Mean(clip[, discard: Int = 0, preset: Int = 0])Where discard is the number of highest and lowest values to drop from the input pixels, and preset is the hard coded IPB weights. The current API does not directly expose weighting to the user, so you must currently use one of the build ins. Also note that discard and preset cannot both be used together.
Median has not (at least in terms of functionality) changed at all in that build above: vapoursynth-median's MedianBlend is actually just taking a constrained "Mean", and is the only one of it's three functions to actually implement the high and low. So you're request is actually only relevant for average.Mean, unless I'm also misunderstanding.
Let me know if v0.4.0-beta works, and you're happy with it, and I'll merge this into master, write some real documentation, and then release v0.4.0 :)
I tried 0.4 but there is no discard param present. I triple checked everything (and also for multiple dlls in my plugins folder)
Here's the output of core.get_plugins()
"eoe-nephren.average": {
"namespace": "average",
"identifier": "eoe-nephren.average",
"name": "vs-average",
"functions": {
"Mean": "clips:clip[]:empty;output_depth:int:opt;",
"Median": "clips:clip[]:empty;"
}
}
Hmmm...
Given that the output_depth parameter is present on Mean, it seems that you are using an older version of the plugin. Have you thoroughly checked everything?
Yes you're right, I had average.dll and vs_average.dll in my folder. But VS did not complain about duplicate namespaces... :-/ very strange.
Just a quick test but it seems a simple Mean() looks best. With discard=4 (or discard=2) the artifacts are more visible. Median() also looks worse than Mean(). Same goes for https://github.com/dubhater/vapoursynth-median/, Mean() looks best to me with my source.
Btw I could upload the files if you want to test yourself.
I've had very similar results overall, I think its mostly going to be a trial and error as to what works best. Don't worry about the files, but thanks for the offer & detailed response :)
I'm still working on a few other possible ideas as well, but for now we'll merge this into master, and I'll finish writing the documentation. Once that's done, if you're happy, I'll close this issue.
I'm just wondering why Mean looks better than Median. In my head it should have been the other way around 😄
It seems that my assumption was simply wrong.
I think in my case is that I have more bad frames in the same place and with Median the "good quality frames" gets ignored bcs they are the extreme values while Mean takes them into account. That's why Mean looks better I think.
MedianBlend was created for cleaning multiple video recordings. It's a slightly different use case.
List of values L { 1, 2, 3, 6, 8, 10, 30, 35, 50 }
Median = 8
Maybe discarding +- X values around Median from L and then calucation Mean values = the good stuff (for my case)? Or maybe complete nonsense...
X = 1
Remove 6,8,10 from L
L { 1, 2, 3, 30, 35, 50 }
Mean(L)
If you are ever bored, you could try it out 😁