Simultaneous channel visualization, Mid/Side representations, and complex-valued inputs
TF3RDL opened this issue · comments
Would be cool to have the feature to display multiple channels simultaneously (instead of having multiple DUI elements or CUI panels just for visualization of multiple channels at once), either displayed in vertical stacks like Spectroscope visualization from WaveLab, mirrored horizontal like foo_uie_vis_channel_spectrum and my own WebAudio recreation of Channel Spectrum panel component, or even combined like dual-combined mode in audioMotion-analyzer
Also, a Mid/Side representation would be nice, but according to foobar2000 SDK, there are 18 channels to convert to M/S representation (M = (L+R)/2 and S = (L-R)/2), which makes doing M/S conversion look complicated when comes to audio with channels more than two like 5.1 and 7.1 surround sound:
channel_front_left = 1<<0
channel_front_right = 1<<1
channel_front_center = 1<<2
channel_lfe = 1<<3
channel_back_left = 1<<4
channel_back_right = 1<<5
channel_front_center_left = 1<<6
channel_front_center_right = 1<<7
channel_back_center = 1<<8
channel_side_left = 1<<9
channel_side_right = 1<<10
channel_top_center = 1<<11
channel_top_front_left = 1<<12
channel_top_front_center = 1<<13
channel_top_front_right = 1<<14
channel_top_back_left = 1<<15
channel_top_back_center = 1<<16
channel_top_back_right = 1<<17
So, channels would make sense to be converted to M/S are:
0 and 1: Front left/right
4 and 5: Back left/right
6 and 7: Front center left/right
9 and 10: Side left/right
12 and 14: Top front left/right
15 and 17: Top back left/right
and what channels wouldn't make sense be converted to M/S (or in other words, better off to be left off) are:
2: Front center
3: LFE
8: Back center
11: Top center
13: Top front center
16: Top back center
therefore, a M/S representation of all 18 defined channels would be (12 to be actually converted to M/S and ignore the redundancy):
0 = FM: Front mid
1 = FS: Front side
4 = BM: Back mid
5 = BS: Back side
6 = FCM: Front center mid
7 = FCS: Front center side
9 = SM: Side mid
10 = SS: Side side
12 = TFM: Top front mid
14 = TFS: Top front side
15 = TBM: Top back mid
17 = TBS: Top back side
What's wrong with having multiple instances? Before you know it someone wants to rearrange the presentation or wants separate parameters for each graph. That's what the layout system and the saveable themes of foobar2000 are for, IMHO.
The channel aggregation is worth looking into but first the stereo channel problem needs to be solved.
What's wrong with having multiple instances? Before you know it someone wants to rearrange the presentation or wants separate parameters for each graph. That's what the layout system and the saveable themes of foobar2000 are for, IMHO.
Yeah, having multiple instances that has its own configs are actually fine, but what I don't like is the hassle of having to change the settings for these two or more instances every time I want to change FFT size, number of bands, and other parameters, and since foobar2000 can support up to 18 channels of surround sound (which is 17.1) and number of channels used for visualization actually varies between audio files and some DSP effects (especially upmixing and downmixing) can change the number of channels when applied, so having simultaneous channel view that would automatically adapts to different channel configurations within a single DUI element/CUI panel would be appreciated
The channel aggregation is worth looking into but first the stereo channel problem needs to be solved.
The problem of discontinuities that led to wonky spectrum when comes to two or more channels as input for this visualization component are mentioned at #5 but I think an oscilloscope view (a plot of FFT input data, which is real-valued anyway) would be appreciated for debugging purposes (e.g. implementation of new window functions) only
What I meant by "combined" in terms of channel spectrum visualization is this:
where blue and red lines are Left/Right channels and green and purple are Mid/Side channels displayed in a single graph and superimposed into one another
Of course, another but better way to implement Mid/Side mode is having two "Channels" parameter; the first one is subtracted by the second one, so this way, it is possible to visualize the difference between front and rear channels and even Center and LFE channels, not just visualization of L/R channels that would be lost if converted to mono
Maybe four "Channels" parameter (the first half is the real input, and the latter half is the imaginary input, so the Mid/Side visualization is still possible after all) for complex-valued FFT visualization like this, which is typically used for I/Q signals in SDR, but idk how would be useful for audio analysis
Of course, the division of the summed sample values before FFT has to be done correctly to have 0dBFS spectral peak on a 0dBFS sine wave when all channels are used