Demos bars and spec not working
davidmartinrius opened this issue · comments
Hello!
How are you doing? I checked out this project and I liked it. It think it can be useful for automated video editing.
I tried all the demos "broken shaderflow {whatever}"
All worked except bars and spec.
When running bars I found a bug at:
BrokenSource/Broken/Optional/BrokenAudio.py
Thread-1 (catch_up_recorder):
Traceback (most recent call last):
File "mydir/.pyenv/versions/3.11.6/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
File "/mydir/.pyenv/versions/3.11.6/lib/python3.11/threading.py", line 982, in run
self._target(*self._args, **self._kwargs)
TypeError: BrokenAudio.catch_up_recorder() got an unexpected keyword argument 'loop'
def start_capture_thread(self) -> None: """Keep recording audio on a separate thread""" BrokenThread.new(self.catch_up_recorder, daemon=True, loop=True)
I am using an Ubuntu22 with python3.11
I just removed loop parameter and no longer showed that error, but the demo still doesn't work.
Both bars an spec shows a black screen in a window. The other ones (default, dynamics, nested and noise) work properly.
When running bars an spec the console just shows this:
│ShaderFlow├┤ 399ms├┤INFO │ ▸ ____ _ _ _____ _
│ShaderFlow├┤ 400ms├┤INFO │ ▸ / || | __ _ _| | ___ _ | | | _____ __
│ShaderFlow├┤ 400ms├┤INFO │ ▸ _ | ' \ / _ |/ _ |/ _ \ '| | | |/ _ \ \ /\ / /
│ShaderFlow├┤ 400ms├┤INFO │ ▸ ) | | | | (| | (| | __/ | | | | | () \ V V /
│ShaderFlow├┤ 400ms├┤INFO │ ▸ |/|| ||_,|_,|_|| || |_|_/ _/_/
│ShaderFlow├┤ 400ms├┤INFO │ ▸
│ShaderFlow├┤ 400ms├┤INFO │ ▸ Made with ❤️ by BrokenSource, Version: (2024.2.6)
│ShaderFlow├┤ 400ms├┤INFO │ ▸ Development version @ Python 3.11.6
│ShaderFlow├┤ 400ms├┤INFO │ ▸
│ShaderFlow├┤ 444ms├┤INFO │ ▸ │ 1├┤Bars │ Dynamically importing (headless) Window class
│ShaderFlow├┤ 445ms├┤INFO │ ▸ │ 1├┤Bars │ Creating Window
│ShaderFlow├┤ 499ms├┤INFO │ ▸ │ 1├┤Bars │ Binding to Window's OpenGL Context
│ShaderFlow├┤ 507ms├┤INFO │ ▸ │ 1├┤Bars │ Finished Window creation
│ShaderFlow├┤ 514ms├┤SUCCESS│ ▸ Found loopback device (Monitor of Built-in Audio Analog Stereo)
│ShaderFlow├┤ 518ms├┤INFO │ ▸ Opening recorder with device (<Loopback Monitor of Built-in Audio Analog Stereo (2 channels)>)
│ShaderFlow├┤ 553ms├┤INFO │ ▸ Making Spectrogram Matrix (20.00Hz - 20000.00Hz) with 1000 bins)
│ShaderFlow├┤ 577ms├┤INFO │ ▸ Making Spectrogram Piano Matrix from notes (D#0 - C#10)
│ShaderFlow├┤ 578ms├┤INFO │ ▸ Making Spectrogram Matrix (19.45Hz - 17739.69Hz) with 119 bins)
│ShaderFlow├┤ 581ms├┤INFO │ ▸ │ 1├┤Bars │ Destroying previous Window
│ShaderFlow├┤ 582ms├┤INFO │ ▸ │ 1├┤Bars │ Dynamically importing (headless) Window class
│ShaderFlow├┤ 582ms├┤INFO │ ▸ │ 1├┤Bars │ Creating Window
│ShaderFlow├┤ 613ms├┤INFO │ ▸ │ 1├┤Bars │ Rebinding to Window's OpenGL Context
│ShaderFlow├┤ 615ms├┤INFO │ ▸ │ 1├┤Bars │ Finished Window creation
│ShaderFlow├┤ 615ms├┤INFO │ ▸ │ 1├┤Bars │ Resizing window to size (1920x1080)
│ShaderFlow├┤ 620ms├┤INFO │ ▸ │ 1├┤Bars │ Changing SSAA to 1.0
│ShaderFlow├┤ 624ms├┤INFO │ ▸ │ 1├┤Bars │ Changing backend to SombreroBackend.GLFW
│ShaderFlow├┤ 625ms├┤INFO │ ▸ │ 1├┤Bars │ Destroying previous Window
│ShaderFlow├┤ 625ms├┤INFO │ ▸ │ 1├┤Bars │ Dynamically importing (glfw) Window class
│ShaderFlow├┤ 627ms├┤INFO │ ▸ │ 1├┤Bars │ Creating Window
│ShaderFlow├┤ 771ms├┤INFO │ ▸ │ 1├┤Bars │ Rebinding to Window's OpenGL Context
│ShaderFlow├┤ 1082ms├┤INFO │ ▸ │ 1├┤Bars │ Implementing file dropping for GLFW
│ShaderFlow├┤ 1083ms├┤INFO │ ▸ │ 1├┤Bars │ Finished Window creation
│ShaderFlow├┤ 1089ms├┤INFO │ ▸ │14├┤SombreroEngine │ Reloading shaders
│ShaderFlow├┤ 1092ms├┤INFO │ ▸ │13├┤SombreroEngine │ Reloading shaders
Please, could you fix it? Or tell me how to do it?
I just want to see all the possibilities of this project to make my own scenes from it.
Thank you!
oops, didn't noticed loop=True was being used somewhere else
I had previously thought to delegate looping functionality to the function a thread is being created itself, but now I see why I did that, slightly cleaner code. That is a fix on the monorepo with the commit 1cc0b72
I just removed loop parameter and no longer showed that error, but the demo still doesn't work.
The demo likely didn't work as the .record() method wasn't being called, well, in a loop. But I also saw some other issues with it, namely on the setup method, which I fixed in b13c298 on this repo
Getting the changes
As it involves two repos, you'll need to git pull on the monorepo and on Projects/ShaderFlow repo
I have written some git automation on the main manager, can you try running broken submodules --pull to see if it pulls the two fix commits, and give a feedback?
Thanks 👍
A Note on audio scenes,
There is a lot of technical debt on the current audio code, and I am planning a full rewrite of it soon. I lazily ported/refurbished research & development code from older attempts to write the project, which took a while to "master"
It's not as good (depending on one's definition) or clean, flexible, it can be a lot better and also needs the option to render a video and use audio files as the audio stream
Regardless of that, I can see two main issues you might face for now:
- Wrong device being opened for recording
You can send a audio.open_device(name="...") parameter on the demo audio scenes on Resources/Scenes/Examples/Basic.py that will be fuzzy searched (similarity check) against all reported devices by the SoundCard package (or change the device on pavucontrol's Recording tab as you're on linux)
- The music bars or spectrogram aren't real time
I had that happening on slower CPU hardware, the implementation can likely be optimized if that's the root issue which will be on the next gen code, I assume you at least have half decent hardware as DepthFlow ran fast :)
Aight, first piece of the Audio puzzle is done, made a safe™, low memory, progressive function to read audio chunks from files or videos (using FFmpeg of course). In my system it took 22.3 seconds to read a 5 GB 2h40m video file with Float 32 Stereo Audio at 44100 Hz and about 160 MB of RAM
I could not be happier with this result comparing to ancient code of mine, which was wayy slower
Still gotta rewrite some audio buffer class and integrate it with Sombrero (the shader engine) plus audio processing. And slightly more scientific spectrogram classes !
Hey, just pushed some, perhaps interesting changes 👀
broken shaderflow visualizer
Spoiler
ShaderFlow can now render audio reactive videos with some input audio up to 16k resolution, any framerate (set the input file on the Basic.py demo scenes file)
Proof of concept code, will do a cleanup, simplification, review and implement other quality of life and interesting modules to have (waveform)
Remember to enable audio ! Sudden stop cuz GitHub file size limit 🙈
2024.2.17.ShaderFlow.Music.Visualizer.Proof.of.Concept.mp4
Awesome! I'll try it!! 😄
Hey @davidmartinrius, I've changed enough of the code and architecture for the original issue be long gone, tested on Linux and Windows the new audio classes, and they seem to be working
Feel free to reopen the issue or send further messages while still closed if there's more issues on the audio processing, I'm yet to give it some more love and architecture enhancements for optimizations
Thank you @Tremeschin