Freenect(2) update thread (Linux/macOS support)
SirLynix opened this issue · comments
Hi,
With the upcoming 0.3 release, I splitted obs-kinect into multiple parts: common stuff like obs plugin registration, depth/infrared/faux green-screen processing, source and general device handling is part of the core but it doesn't know about Kinect devices (or even windows). Kinect v1/v2 handling is done in separate dll which are loaded by obs-kinect at startup.
This was made to add support for Kinect v1 without requiring the install of both v1 and v2 runtimes. And also to support Linux/macOS.
Obviously, Microsoft Kinect SDKs don't work on Linux/macOS, hopefully projects like libfreenect and libfreenect2 allow the use of Kinect on these systems.
Since obs-kinect device handling is done in separate libraries, it's possible to add libfreenect support kinda easily.
I tried both of these libraries on Windows (since they do have Windows support as well) for testing and it's working well!
freenect (v1):
freenect2 (v2):
However since libfreenect(2) needs raw USB commands, it requires to install a custom USB driver (like UsbDk). Hopefully this doesn't conflict with Kinect v2 (I can use libfreenect2 and Kinect SDK v2.0 without problem), but well, for Kinect v1 it's a bit more like hell. You have to replace the Kinect USB driver (using Zadig) for libfreenect to work which prevents Kinect SDK v1.8 to work, and it took me one hour to reinstall the official Kinect drivers, so yeah, I don't think I will release an obs-kinect libfreenect backend on Windows because of that (since it doesn't seem to do things you cannot do with official Kinect v1.8 runtime, except maybe for led control?).
As for Kinect v2, libfreenect2 does add more control, especially with the RGB camera color settings (things like exposure mode, white balance, gain, and such), so I think I'll support it on Windows too (even though I hope to be able to do that as well with the official SDK backend, using this).
Important note
Both libfreenect libs are really cool, but they are lacking some higher-level stuff the Kinect SDK does, like body/skeletal detection, or dedicated background removal (like Kinect SDK v1.8 does). It doesn't mean you won't be able to have background removal on Linux/macOS but it means you won't be able to use body informations for that and there's not much I can do about that. (except implementing body detecting but it requires some skill I don't have).
I will be able to compile the plugin on Linux, or even test it without problems, but I won't be able to do that with macOS. So if you have a mac and are willing to help, please let me know!
Also, as the 0.3 release is already a big one, Linux/macOS support won't be part of it (but I'll try to do it asap).
TL;DR: Linux/macOS support requires an open-source backend since official Microsoft SDKs are for Windows. There are backend for Kinect v1 and v2 but they don't do higher-level processing like body information. Things like body detection/body-based greenscreen filtering won't work on Linux/macOS because of that.
@SirLynix I can do Mac builds and testing for you. I have both a Kinect v2 and a v1.
Thanks! I have little experience with macOS compilation, I'll probably need some help!
Hello :)
it seems that you only changed "include" and "src" files.
So how can we update the software? Which files must be copied and pasted to where?
Thanks in advance.
W10 and Kinect v2
I am really sorry but i am not a coder. I only know node based coding. I am really far away from visual studio. Is there any quick istall file or something like that? Or do you consider to make one for end users like me in future? Thanks :)
You can download the 0.2 release in the releases section of GitHub. The 0.3 binaries will be downloadable soon (I will make a release candidate for them).
As for the Kinect v2, there's not much difference between 0.2 and 0.3 for now, except for some performance improvement.
I've tested libfreenect on my current machine (archlinux install) and it works pretty well. Is there any reasonably straightforward way to build this with libfreenect as it currently stands? I'd be willing to go through the effort and test it on my system, this is something i've been wanting for a while now!
Hi!
From what I saw libfreenect(2) should be kinda easy to use, and Linux support is my next big task. It's just that I have a lot of work to do for other projects (and I have a relatively costly hobby called eating 😄 ) and this is not at the top of my todo list for now.
However a lot of the work that went into 0.3 (that will be 1.0 when out) was to make obs-kinect core portable, and leave Windows/Kinect SDK specific stuff in some backend code. All that to say there's not much work remaining in order to have this plugin on Linux!
Fantastic! If you need someone to test, please feel free to find a way to contact me haha.
I am trying to use Kinect 2 as a webcam on Mac (using Slack). Using Libfreenect2 I can get the RGB stream through their example app (Protonect) or code but there is no straightforward way to expose Kinect as a webcam to other software.
Since OBS can act a virtual webcam on Mac having this plugin working would be highly useful.
Hello!
I have to agree, I would love to see a working macOS Plugin. I've been looking for a solution to use my Kinect v2 via libfreenect2 as a webcam too for quite some time now, this is by far the most promising solution. Great work so far!
Just a quick message to tell you guys I'm not forgetting about Linux and macOS, I'm just working on several things at once (a man has to eat). My next big task (and the only remaining missing feature for 1.0) is Linux and macOS support. 😄
First KinectV2 color image captured using the WIP freenect2 background!
I'm testing it on Windows for now but this backend is made for Linux and macOS users 😄
I'm also switching from premake to xmake as a build tool for its handling of dependencies, and planning to setup a CI (with nightly builds).
I managed to make the fake greenscreen work with libfreenect2. The results seems worse than with the official Windows SDK (it may be because of some options I still have to tweak).
That is cool. Personally I am happy with having greenscreen only on Windows. I only need it as a webcam on mac.
Do you know if the Linux version will have the same impact on system performance than on Windows? I dropped the Kinect atm on Windows because of that.
I hope it will be less, as at least the Kinect windows service will not be there?!
Otherwise I will sell my Kinect due to lack of availability of graphics cards at normal prices atm
It depends on the Kinect model.
One of the benefit of Freenect2 (kinect for xbox one) is that it is able to exploit either the CPU or GPU to perform depth processing. I don't know if it will be faster than what the official SDK does but at least there will be more control.
I don't really know about Freenect (kinect for 360), as it's a completely different API, but I think it will be handled in a smarter way (the official Windows SDK tries to use all CPU threads at once).
As for Azure Kinect, I don't think there will be a significant difference (as it's the same SDK). Even though there maybe a difference between Windows and Linux/macOS on low-level handling.
So the real answer here is: I don't know yet, it requires some testing.
I managed to setup a CI which will build Linux plugins files (and eventually Windows and macOS) with a preliminary support for Kinect v2 (xbox one) through freenect2, and Azure Kinect support through Kinect for Azure SDK.
If you're feeling adventurous, you can download the Linux plugins files here (click on the last workflow that ran on master and download the ubuntu-latest-x64-releasedbg artifacts).
You'll still need a few other files (currently missing from this package), so you'll need to download the latest Windows nightlies to get theses files (please do it only if you know what you're doing).
This is not tested.
If you went into the troubles of trying this, please let me know if and how it worked for you (even if it didn't)!
Kinect v1 (xbox 360) and macOS users, don't worry. You'll be able to try it out soon!
I improved packaging support for Windows and Linux, which means the Linux package now has obs-kinect data (shaders and locales).
I'm wondering, what would be the best way to distribute obs-kinect on Linux? I've seen that some plugins (such as obs-websocket) build a .deb package which automatically installs the plugin files at the right place, which is something I can do, but doesn't that excludes non-debian based Linux distributions? What about them? Is obs studio even available/used on such distributions?
About mac builds, things seems easier on a technical level but would require me to pay for an Apple Developer Certificate annually. I need to think about this, and if there's another way.
I'm wondering, what would be the best way to distribute obs-kinect on Linux? I've seen that some plugins (such as obs-websocket) build a .deb package which automatically installs the plugin files at the right place, which is something I can do, but doesn't that excludes non-debian based Linux distributions? What about them? Is obs studio even available/used on such distributions?
Yes, if it´s a .deb package it´s only for debian based distros. AFAIK advanced users could extract that package and copy to the right places on their distro, too. IIRC I´ve done that once but not sure. I tried Solus, elementaryOS and am now with Manjaro (arch based). For all theses distros there is OBS "natively" available through their package managers.
Compiling it for one/serveral formats would limit it to these distros. Maybe at least give a manual how to install from scratch into most common distros/bases (don´t know the right term) (deb, arch). There are also distro-wide packages like flat, snap and some third I can´t remember. Though I´m not sure if this will work only together with OBS installed using this packages, too.
First image of obs-kinect on Linux!
I installed Ubuntu 20.04 and went into some troubles but finally found a way to make it work.
From what I observed, freenect2 works well, and is able to use my KinectV2 without any issue, except for the root permissions (which seems to be easy to fix, see OpenKinect/libfreenect2#870 ).
The Azure backend works too, but for some weird reason I couldn't retrieve the depth or infrared streams, probably something on my side.
The main remaining issue which remains is now a rpath one, I'll explain:
OBS-Studio loads obs-kinect which then loads its backends (obs-kinect-freenect2 for example), but this one is linked against obs-kinect and it seems Linux wants to open libobs-kinect.so back from there, except it just can't be found since libobs-kinect belongs to the plugin folder.
Setting the rpath to $origin doesn't help, as the origin here is OBS, and I can't set it to something like $origin/../../obs-plugins/64bit (which I did to test locally) because this path isn't guaranteed.
I see some solutions to fix this:
- Find some rpath trick to retrieve libobs-kinect.so.
- Split libobs-kinect.so in two files, set one next to obs executable and the other in obs-plugins folder (and make the backend linked to the first one, so no direct link between the plugin and its backend).
- Use static linking.
- Merge obs-kinect backend code into the main plugin and dynamically load backend dependencies.
I'm not sure if 1) is even possible, 3) is ugly and 4) is difficult to implement (as the libraries used are not really meant for that), so I think I'll go with 2).
Oh and I had to fix a shader on Linux, which now doesn't compile on Windows, yay.
I managed to work around theses problems and even make my Azure Kinect work on Linux (with depth-based blur!), I was just lacking Microsoft proprietary depth engine.
If the last commit compiles on the CI, it should be working for Linux users.
I'm wondering, what would be the best way to distribute obs-kinect on Linux? I've seen that some plugins (such as obs-websocket) build a .deb package which automatically installs the plugin files at the right place, which is something I can do, but doesn't that excludes non-debian based Linux distributions? What about them? Is obs studio even available/used on such distributions?
obs studio is available on most mainstream distros (debian, arch, gentoo probably even) and your "duty" as the developer in the typical linux packaging world is just really just tagging releases in github. Then those builds are reproducible for package maintainers in theory.
Do you know a OBS Studio plugins for Linux which does everything that should be done for this platform? The only example I've seen is obs-websocket which only provides a .deb package for Debian-based distributions.
On my previous Manjaro installation I had the "NDI" plugin installed. Oh and on my laptop with Solus OS, too.
Don't know if this helps you.
How did you install them? I can only find debian packages on their Github page
Honestly I can't remember. I just did a lookup and at least on Manjaro there are packages available in the AUR from where it can be build on demand. For Solus it COULD be similar.
But iirc I had to adapt some files/paths at least on one of the two OS. Though I'm really really not sure about. I do not use OBS on Linux atm so I'm sorry I can not give you more information about it.
I'm pretty sure though, that it depends on how one installs OBS.
Theses app/snap/flat packages are (all?) sandboxed and therefore plugins may be not compatible/different to install.
Thanks for the info. I think I will just bother distributing a .deb package for now (once Linux support is complete) and we'll see for other distributions after that.
Any updates for the macOS users?
Hi.
In theory obs-kinect master should already support macOS, but I think I need an Apple computer to build it myself, so I should try to setup a CI action for that.
Sorry for the wait, it's because of my lack of experience with mac platform.
First image of the Freenect backend (Kinect v1 for Linux/macOS), it's happening!
Is there a branch that we can test already ?
Hi!
You can either build the plugin yourself or download the latest Linux nightlies.
Freenect2 (Kinectv2) and Azure (Kinectv3) support should be fine, Freenect (Kinectv1) should work for color. It's a bit difficult for me to work on it at the moment but I'd love to have feedbacks.
I just tried the plugin.
It does not seem to work, but I'm not sure it's the plugin fault.
For context : I have a Kinect v1 (model 1414). It is not detected by the plugin, or anywhere else except freenect-glview actually...
Here is what lsusb gives me :
Bus 001 Device 107: ID 045e:02ae Microsoft Corp. Xbox NUI Camera
Bus 001 Device 105: ID 045e:02b0 Microsoft Corp. Xbox NUI Motor
Bus 001 Device 106: ID 045e:02ad Microsoft Corp. Xbox NUI Audio
If you need help to test or debug anything, I'm here to help :-) I never developed a plugin for OBS, though.
I think I might know what causes this, I'll try to commit a fix soon!
Same issue here, I installed the .so files in /usr/lib/obs-plugins and see that they are being loaded by the obs-studio. I got some errors on missing libraries which I added all to the folder. After this the error messages disappeared.
But I see no option to select the Kinect v1.Does anyone know how I can check if the plugin is installed fine? Also the build artifacts were removed, so if someone could trigger a rebuild, that would be nice.
I was looking to implement this idea of using libfreenect and pipe this to v4l2 to use the depth functionality in any app. Pretty cool this obs app.
Check your OBS logs to see if the plugin is correctly loaded.
I just triggered a new build, it should be good in about 10min
I see no mention of the plugin files in the logs, however I do see some attempt in strace to load the .so files.
Can they all go in the same dir or do I need to keep the subfolder structure?
currently I have obs-kinect-freenect.so, libobs-kinectcore.so and obs-kinect-freenect.so in /usr/lib/obs-plugins.
Honestly I'm not sure at all of where obs will fetch plugins file, I did make it work one time on Ubuntu but I'm not using Linux much, especially not graphical Linux.
Build failed because of some link order issue, which is weird, I reported it to xmake author.
Ah ok, just wanted to know whether the .so files rely on the directory structure that comes with the zip file. But basically if the plugin is loaded I should be able to add it as a source even if there is no kinect connected? If that is the case then I guess the plugin is not loaded properly (as I see no extra source in the dropdown)
Yes, if the plugins loads you should be able to create a Kinect source, even if it failed to load any backend.
If you don't see a Kinect source in the list, then the plugin itself failed to load.
Thanks to xmake author, the bug was quickly fixed and artifacts are now available for Linux.
I tried the latest build and I still don't see the Kinect v1 in the devices list.
However, I can use the built-in video capture of OBS to use the Kinect as a camera - without much configuration options.
Hence, maybe this is why the plugin does not detect the Kinect ? (I actually doubt it, but if it can help you find the problem...)
[EDIT] Just realized I can also get the IR camera stream from the built-in video capture.
This seems pretty normal and isn't an issue, do you have any mention to obs-kinect in OBS logs?
hello, i'm trying to use kinect v1 with this plugin on Ubuntu 20.04 focal, with libfreenect installed from repositories and obs from ppa:
freenect/focal,now 1:0.5.3-2 amd64 [installato]
libreria per accedere ai dispositivi Kinect - metapacchetto
libfreenect-bin/focal,now 1:0.5.3-2 amd64 [installato, automatico]
library for accessing Kinect device -- utilities and samples
libfreenect-demos/focal 1:0.5.3-2 amd64
library for accessing Kinect device -- dummy package
libfreenect-dev/focal,now 1:0.5.3-2 amd64 [installato, automatico]
library for accessing Kinect device -- development files
libfreenect-doc/focal,focal,now 1:0.5.3-2 all [installato, automatico]
library for accessing Kinect device -- documentation
libfreenect0.5/focal,now 1:0.5.3-2 amd64 [installato, automatico]
library for accessing Kinect device
obs-studio/focal,now 27.1.3-0obsproject1~focal amd64 [installato]
OBS Studio for Ubuntu
then downloaded the releasedbg from here https://github.com/SirLynix/obs-kinect/actions/runs/1539175629
and put file using this folder structure:
tree -L 3 .config/obs-studio/
.config/obs-studio/
├── basic
│ ├── profiles
│ └── scenes
├── global.ini
├── logs
├── plugin_config
│ ├── rtmp-services
│ │ ├── meta.json
│ │ ├── package.json
│ │ ├── services.json
│ │ └── twitch_ingests.json
│ └── text-freetype2
├── plugins
│ └── obs-kinect
│ ├── alpha_mask.effect
│ ├── color_multiplier.effect
│ ├── gaussian_blur.effect
│ ├── greenscreen_filter.effect
│ ├── locale
│ ├── texture_lerp.effect
│ └── visibility_mask.effect
└── profiler_data
and:
tree -L 3 /usr/lib/obs-plugins
/usr/lib/obs-plugins/
├── cef_100_percent.pak
├── cef_200_percent.pak
├── cef_extensions.pak
├── cef.pak
├── chrome-sandbox
├── decklink-captions.so
├── decklink-ouput-ui.so
├── devtools_resources.pak
├── frontend-tools.so
├── icudtl.dat
├── image-source.so
├── libcef.so
├── libEGL.so
├── libGLESv2.so
├── libobs-kinectcore.so
├── linux-alsa.so
├── linux-capture.so
├── linux-decklink.so
├── linux-jack.so
├── linux-pulseaudio.so
├── linux-v4l2.so
├── locales
├── [...]
├── obs-browser-page
├── obs-browser.so
├── obs-ffmpeg.so
├── obs-filters.so
├── obs-kinect-azuresdk.so
├── obs-kinect-freenect2.so
├── obs-kinect-freenect.so
├── obs-kinect.so
├── obs-libfdk.so
├── obs-outputs.so
├── obs-transitions.so
├── obs-vst.so
├── obs-x264.so
├── rtmp-services.so
├── snapshot_blob.bin
├── swiftshader
│ ├── libEGL.so
│ └── libGLESv2.so
├── text-freetype2.so
├── v8_context_snapshot.bin
└── vlc-video.so
I can see the plugin in the menu:
but it crashes when I try to add it:
strace -o obstrace.txt obs
Attempted path: share/obs/obs-studio/locale/en-US.ini
Attempted path: /usr/share/obs/obs-studio/locale/en-US.ini
Attempted path: share/obs/obs-studio/locale.ini
Attempted path: /usr/share/obs/obs-studio/locale.ini
Attempted path: share/obs/obs-studio/locale/it-IT.ini
Attempted path: /usr/share/obs/obs-studio/locale/it-IT.ini
info: Using preferred locale 'it-IT'
Attempted path: share/obs/obs-studio/themes/Dark.qss
Attempted path: /usr/share/obs/obs-studio/themes/Dark.qss
info: CPU Name: Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz
info: CPU Speed: 987.230MHz
info: Physical Cores: 2, Logical Cores: 4
info: Physical Memory: 7866MB Total, 1865MB Free
info: Kernel Version: Linux 5.4.0-91-generic
info: Distribution: "Ubuntu" "20.04"
info: Session Type: x11
info: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.20.13
info: Portable mode: false
Attempted path: share/obs/obs-studio/themes/Dark/no_sources.svg
Attempted path: /usr/share/obs/obs-studio/themes/Dark/no_sources.svg
QMetaObject::connectSlotsByName: No matching signal for on_tbar_position_valueChanged(int)
QMetaObject::connectSlotsByName: No matching signal for on_actionShowTransitionProperties_triggered()
QMetaObject::connectSlotsByName: No matching signal for on_actionHideTransitionProperties_triggered()
info: OBS 27.1.3 (linux)
info: ---------------------------------
info: ---------------------------------
info: audio settings reset:
samples per sec: 48000
speakers: 2
info: ---------------------------------
info: Initializing OpenGL...
info: Loading up OpenGL on adapter Intel Mesa Intel(R) HD Graphics 5500 (BDW GT2)
info: OpenGL loaded successfully, version 4.6 (Core Profile) Mesa 21.0.3, shading language 4.60
info: ---------------------------------
info: video settings reset:
base resolution: 640x240
output resolution: 640x240
downscale filter: Bicubic
fps: 30/1
format: NV12
YUV mode: 709/Partial
info: NV12 texture support not available
info: Audio monitoring device:
name: Predefinito
id: default
info: ---------------------------------
warning: Failed to load 'en-US' text for module: 'decklink-captions.so'
warning: Failed to load 'en-US' text for module: 'decklink-ouput-ui.so'
libDeckLinkAPI.so: cannot open shared object file: No such file or directory
warning: A DeckLink iterator could not be created. The DeckLink drivers may not be installed
info: No blackmagic support
info: [obs-browser]: Version 2.16.3
info: [obs-browser]: CEF Version 87.1.12+g03f9336+chromium-87.0.4280.88
error: os_dlopen(libnvidia-encode.so.1->libnvidia-encode.so.1): libnvidia-encode.so.1: cannot open shared object file: No such file or directory
info: FFMPEG VAAPI supported
error: os_dlopen(/usr//lib/obs-plugins/obs-kinect-azuresdk.so->/usr//lib/obs-plugins/obs-kinect-azuresdk.so): libk4a.so.1.4: cannot open shared object file: No such file or directory
warning: Module '/usr//lib/obs-plugins/obs-kinect-azuresdk.so' not loaded
warning: Failed to load 'en-US' text for module: 'obs-kinect.so'
error: os_dlopen(./obs-kinect-azuresdk->./obs-kinect-azuresdk.so): ./obs-kinect-azuresdk.so: cannot open shared object file: No such file or directory
error: os_dlopen(./obs-kinect-freenect->./obs-kinect-freenect.so): ./obs-kinect-freenect.so: cannot open shared object file: No such file or directory
error: os_dlopen(./obs-kinect-freenect2->./obs-kinect-freenect2.so): ./obs-kinect-freenect2.so: cannot open shared object file: No such file or directory
error: os_dlopen(./obs-kinect-sdk10->./obs-kinect-sdk10.so): ./obs-kinect-sdk10.so: cannot open shared object file: No such file or directory
error: os_dlopen(./obs-kinect-sdk20->./obs-kinect-sdk20.so): ./obs-kinect-sdk20.so: cannot open shared object file: No such file or directory
info: VLC found, VLC video source enabled
info: ---------------------------------
info: Loaded Modules:
info: vlc-video.so
info: text-freetype2.so
info: rtmp-services.so
info: obs-x264.so
info: obs-vst.so
info: obs-transitions.so
info: obs-outputs.so
info: obs-libfdk.so
info: obs-kinect.so
info: obs-filters.so
info: obs-ffmpeg.so
info: obs-browser.so
info: linux-v4l2.so
info: linux-pulseaudio.so
info: linux-jack.so
info: linux-decklink.so
info: linux-capture.so
info: linux-alsa.so
info: image-source.so
info: frontend-tools.so
info: decklink-ouput-ui.so
info: decklink-captions.so
info: ---------------------------------
info: ==== Startup complete ===============================================
info: All scene data cleared
info: ------------------------------------------------
info: pulse-input: Server name: 'pulseaudio 13.99.1'
info: pulse-input: Audio format: s16le, 44100 Hz, 2 channels
info: pulse-input: Started recording from 'alsa_output.pci-0000_00_1b.0.analog-stereo.monitor'
info: [Loaded global audio device]: 'Audio del desktop'
ALSA lib confmisc.c:767:(parse_card) cannot find card 'Camera'
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such device
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such device
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such device
ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such device
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM front:CARD=Camera,DEV=0
error: alsa-input: Failed to open 'front:CARD=Camera,DEV=0': No such device
info: v4l2-input: Start capture from /dev/video2
error: v4l2-input: Unable to set input 0
error: v4l2-input: Initialization failed
info: v4l2-input: Start capture from /dev/video1
info: v4l2-input: Input: 0
info: v4l2-input: Resolution: 320x240
info: v4l2-input: Pixelformat: YUYV
info: v4l2-input: Linesize: 640 Bytes
info: v4l2-input: Framerate: 30.00 fps
info: v4l2-input: /dev/video1: select timeout set to 166666us (5x frame periods)
info: Switched to scene 'nuova'
info: ------------------------------------------------
info: Loaded scenes:
info: - scene 'stereovision':
info: - source: 'l' (v4l2_input)
info: - source: 'r' (v4l2_input)
info: - source: 'mic' (alsa_input_capture)
info: - scene 'nuova':
info: ------------------------------------------------
info: adding 42 milliseconds of audio buffering, total audio buffering is now 42 milliseconds (source: Audio del desktop)
ALSA lib confmisc.c:767:(parse_card) cannot find card 'Camera'
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such device
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such device
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such device
ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such device
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM front:CARD=Camera,DEV=0
error: alsa-input: Failed to open 'front:CARD=Camera,DEV=0': No such device
Attempted path: share/obs/obs-studio/images/overflow.png
Attempted path: /usr/share/obs/obs-studio/images/overflow.png
ALSA lib confmisc.c:767:(parse_card) cannot find card 'Camera'
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such device
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such device
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such device
ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such device
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM front:CARD=Camera,DEV=0
error: alsa-input: Failed to open 'front:CARD=Camera,DEV=0': No such device
ALSA lib confmisc.c:767:(parse_card) cannot find card 'Camera'
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such device
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such device
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such device
ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such device
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM front:CARD=Camera,DEV=0
error: alsa-input: Failed to open 'front:CARD=Camera,DEV=0': No such device
terminate called after throwing an instance of 'std::runtime_error'
what(): failed to create effect: shader error
Aborted (core dumped)
EDIT:
ok..apparently obs is also looking in /usr/share/obs/obs-plugins/
cat obstrace.txt | grep kinect
stat("/usr//lib/obs-plugins/libobs-kinectcore.so", {st_mode=S_IFREG|0664, st_size=51960, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect-azuresdk.so", {st_mode=S_IFREG|0664, st_size=48096, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect-freenect2.so", {st_mode=S_IFREG|0664, st_size=1347992, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect-freenect.so", {st_mode=S_IFREG|0664, st_size=292136, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect.so", {st_mode=S_IFREG|0664, st_size=97720, ...}) = 0
access("/usr/share/obs/obs-plugins/libobs-kinectcore", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/libobs-kinectcore.so", O_RDONLY|O_CLOEXEC) = 24
access("/usr/share/obs/obs-plugins/obs-kinect-azuresdk", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect-azuresdk.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect-freenect2", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect-freenect2.so", O_RDONLY|O_CLOEXEC) = 25
openat(AT_FDCWD, "/usr//lib/obs-plugins/libobs-kinectcore.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect-freenect", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect-freenect.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect/locale/en-US.ini", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "./obs-kinect-azuresdk.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-az"..., 152) = 152
openat(AT_FDCWD, "./obs-kinect-freenect.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-fr"..., 152) = 152
openat(AT_FDCWD, "./obs-kinect-freenect2.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-fr"..., 155) = 155
openat(AT_FDCWD, "./obs-kinect-sdk10.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-sd"..., 143) = 143
openat(AT_FDCWD, "./obs-kinect-sdk20.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-sd"..., 143) = 143
stat("/home/bitflow/.config/obs-studio/plugins/obs-kinect-freenect", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
access("/home/bitflow/.config/obs-studio/plugins/obs-kinect-freenect/bin/64bit/obs-kinect-freenect.so", F_OK) = 0
access("/home/bitflow/.config/obs-studio/plugins/obs-kinect-freenect/data", F_OK) = 0
openat(AT_FDCWD, "/home/bitflow/.config/obs-studio/plugins/obs-kinect-freenect/bin/64bit/obs-kinect-freenect.so", O_RDONLY|O_CLOEXEC) = 28
write(1, "info: obs-kinect.so\n", 24) = 24
write(15, "00:26:25.084: obs-kinect.so\n", 32) = 32
access("/usr/share/obs/obs-plugins/obs-kinect/color_multiplier.effect", F_OK) = -1 ENOENT (No such file or directory)
It looks like obs-kinect-freenect.so, and locale/shader files are not where they should be. If I recall correctly obs-kinect-freenect.so should be next to obs executable, dunno for locale/shaders files.
obs executable is in /usr/bin but dynamically linked libraries .so usually go in /usr/lib
EDIT:
and /usr/share it is usually used for resources
Ok now the kinect window opens atlist, but kinect is not detected...this is what I have:
tree -L 3 /usr/share/obs/obs-plugins/obs-kinect/
/usr/share/obs/obs-plugins/obs-kinect/
├── alpha_mask.effect
├── color_multiplier.effect
├── gaussian_blur.effect
├── greenscreen_filter.effect
├── locale
│ ├── cs-CZ.ini
│ ├── de-DE.ini
│ ├── en-US.ini
│ ├── fr-FR.ini
│ ├── ko-KR.ini
│ └── pl-PL.ini
├── texture_lerp.effect
└── visibility_mask.effect
tree output is edited here so its more clear:
/usr/lib/obs-plugins/
├── libobs-kinectcore.so
├── obs-kinect-azuresdk.so
├── obs-kinect-freenect2.so
├── obs-kinect-freenect.so
├── obs-kinect.so
cat obstrace.txt | grep kinect
stat("/usr//lib/obs-plugins/libobs-kinectcore.so", {st_mode=S_IFREG|0664, st_size=51960, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect-azuresdk.so", {st_mode=S_IFREG|0664, st_size=48096, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect-freenect2.so", {st_mode=S_IFREG|0664, st_size=1347992, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect-freenect.so", {st_mode=S_IFREG|0664, st_size=292136, ...}) = 0
stat("/usr//lib/obs-plugins/obs-kinect.so", {st_mode=S_IFREG|0664, st_size=97720, ...}) = 0
access("/usr/share/obs/obs-plugins/libobs-kinectcore", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/libobs-kinectcore.so", O_RDONLY|O_CLOEXEC) = 24
access("/usr/share/obs/obs-plugins/obs-kinect-azuresdk", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect-azuresdk.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect-freenect2", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect-freenect2.so", O_RDONLY|O_CLOEXEC) = 25
openat(AT_FDCWD, "/usr//lib/obs-plugins/libobs-kinectcore.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect-freenect", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect-freenect.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect", F_OK) = 0
openat(AT_FDCWD, "/usr//lib/obs-plugins/obs-kinect.so", O_RDONLY|O_CLOEXEC) = 25
access("/usr/share/obs/obs-plugins/obs-kinect/locale/en-US.ini", F_OK) = -1 EACCES (Permission denied)
openat(AT_FDCWD, "./obs-kinect-azuresdk.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-az"..., 152) = 152
openat(AT_FDCWD, "./obs-kinect-freenect.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-fr"..., 152) = 152
openat(AT_FDCWD, "./obs-kinect-freenect2.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-fr"..., 155) = 155
openat(AT_FDCWD, "./obs-kinect-sdk10.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-sd"..., 143) = 143
openat(AT_FDCWD, "./obs-kinect-sdk20.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
write(2, "error: os_dlopen(./obs-kinect-sd"..., 143) = 143
write(1, "info: obs-kinect.so\n", 24) = 24
write(15, "02:44:02.168: obs-kinect.so\n", 32) = 32
access("/usr/share/obs/obs-plugins/obs-kinect/color_multiplier.effect", F_OK) = 0
openat(AT_FDCWD, "/usr/share/obs/obs-plugins/obs-kinect/color_multiplier.effect", O_RDONLY) = 34
access("/usr/share/obs/obs-plugins/obs-kinect/gaussian_blur.effect", F_OK) = 0
openat(AT_FDCWD, "/usr/share/obs/obs-plugins/obs-kinect/gaussian_blur.effect", O_RDONLY) = 34
access("/usr/share/obs/obs-plugins/obs-kinect/greenscreen_filter.effect", F_OK) = 0
openat(AT_FDCWD, "/usr/share/obs/obs-plugins/obs-kinect/greenscreen_filter.effect", O_RDONLY) = 34
access("/usr/share/obs/obs-plugins/obs-kinect/gaussian_blur.effect", F_OK) = 0
access("/usr/share/obs/obs-plugins/obs-kinect/texture_lerp.effect", F_OK) = 0
openat(AT_FDCWD, "/usr/share/obs/obs-plugins/obs-kinect/texture_lerp.effect", O_RDONLY) = 34
access("/usr/share/obs/obs-plugins/obs-kinect/texture_lerp.effect", F_OK) = 0
access("/usr/share/obs/obs-plugins/obs-kinect/visibility_mask.effect", F_OK) = 0
openat(AT_FDCWD, "/usr/share/obs/obs-plugins/obs-kinect/visibility_mask.effect", O_RDONLY) = 34
access("/usr/share/obs/obs-plugins/obs-kinect/alpha_mask.effect", F_OK) = 0
openat(AT_FDCWD, "/usr/share/obs/obs-plugins/obs-kinect/alpha_mask.effect", O_RDONLY) = 34
It looks like your locales are still not at the right place, same goes for libobs-kinectcore.so and obs-kinect-freenect(2).so, which if I recall correctly should be next to obs executable (I'm sorry it's been a while since I tested this on Linux).
@b1tfl0w I have gotten the libfreenect to load by starting obs from the /usr/lib/obs-plugins working dir. It was otherwise complaining with following message
14680 write(2, "error: os_dlopen(./obs-kinect-fr"..., 152error: os_dlopen(./obs-kinect-freenect->./obs-kinect-freenect.so): ./obs-kinect-freenect.so: cannot open shared object file: No such file or directory
while in the logging it did seem to pick it up fine before. Beside this you also need to add libfreenect.so files to LD_LIBRARY_PATH. Not sure if this is needed if you install it through a package manager.
export LD_LIBRARY_PATH=/usr/local/lib
Sorry about this guys, I honestly don't know how to handle this loading properly on OBS. As far as I recall obs-kinect.so (plugin file) should be in the obs-plugins folder and other .so should be next to obs executable (because they're loaded with a relative path).
Maybe loading them with a normal path and putting them in /usr/local/lib would be better.
indeed so you can also start obs from the .so folder as it will then pick up the files with the relative path from there. I am able to see color / depth views now using libfreenect, the background blur is also working when I use depth.
But when I switch to color the greenscreen stops working. This seems to be a libfreenect issue as I see quite some messages like the following
info: [obs-kinect] [freenect] freenect notice: [Stream 70] Lost too many packets, resyncing...
warning: [obs-kinect] [freenect] freenect warning: [Stream 70] Expected max 1748 data bytes, but got 1908. Dropping...
info: [obs-kinect] [freenect] freenect notice: [Stream 70] Invalid magic ffff
info: [obs-kinect] [freenect] freenect notice: [Stream 70] Invalid magic ffff
info: [obs-kinect] [freenect] freenect notice: [Stream 70] Invalid magic ffff
This could be a local USB bandwith issue, I'm looking into this and trying to recompile libfreenect with some optimizations now
Ok I'm not sure if this is the issue for the greenscreen not showing in color mode. The messages also appear if I open the depth camera. I can also open both color/depth side by side, but alas no greenscreen on the color version.
This is on me, the obs-kinect-freenect backend doesn't output color-to-depth info yet, mostly because it's a bit difficult to do with libfreenect (which seems to only provide the opposite: depth-to-color). I'll try to resume working on this backend for you 🙂
I didn't really know whether this was a bug on my side, but if it's a feature request, I'd be willing to add sth to your sponsor dashboard if you get it working ;)
I can confirm that setting the obs working directory to /usr/lib/obs-plugins makes it work..
Got the first result, once CI is done compiling it you should be able to test it.
Also please check your OBS locales, if you see "ObsKinect.Device" and such instead of "Kinect device", it means obs-kinect locales are not in the right folder.
@SirLynix I have installed your latest build now, the obs-kinect-freenect.so seems to be a bit smaller (maybe because I downloaded releasedbg this time).
It crashes now with this error
obs: symbol lookup error: /usr//lib/obs-plugins/obs-kinect-freenect.so: undefined symbol: freenect_convert_packed_to_16bit
did I miss something?
Did you install the libfreenect.so included with this build? It is required as it's a custom version
Pulled the version from your account and installed it, works great now. I've added something in paypal, I wanted to give you a one off donation but will keep this one for two months as a reward.
Thanks a lot! I will try to improve the freenect (and Linux) backend in the next few days, as it's still lacking some features currently available on Windows (such as elevation control).
HI @SirLynix, thanks for your excellent work. I've managed to install the latest build, and I can successfully use Depth and Infrared streams:
However, the Color stream is not visible for me:
It seems that this issue is a problem with TurboJPEG, due to the system version of libjpeg:
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 14 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @4:5 with serial 126467240147
[Info] [Freenect2Impl] found 1 devices
[Error] [TurboJpegRgbPacketProcessorImpl] Failed to initialize TurboJPEG decompressor! TurboJPEG error: 'Wrong JPEG library version: library is 80, caller expects 62'
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[...]
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
info: [obs-kinect] [freenect2] Kinect active sources: Color
[...]
[Info] [CpuDepthPacketProcessor] avg. time: 67.6648ms -> ~14.7787Hz
[Info] [DepthPacketStreamParser] 30 packets were lost
[Info] [DepthPacketStreamParser] 30 packets were lost
[Info] [DepthPacketStreamParser] 30 packets were lost
[Info] [DepthPacketStreamParser] 30 packets were lost
As I understand it, you provide a custom libfreenect implementation. However, using the CpuDepthPacketProcessor brings latency and lost packets.
When I run Protonect compiled from libfreenect2, it shows the color stream:
In my case, it employs the system version of TurboJPEG, and uses OpenGLDepthPacketProcessor that offers noticeable performance gains:
[Info] [Freenect2Impl] 14 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @4:5 with serial 126467240147
[Info] [Freenect2Impl] found 1 devices
libva info: VA-API version 1.12.0
libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
[Error] [VaapiRgbPacketProcessorImpl] vaInitialize(display, &major_ver, &minor_ver): unknown libva error
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Debug] [Freenect2DeviceImpl] status 0x090000: 9821
[Debug] [Freenect2DeviceImpl] status 0x090000: 9823
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
device serial: 126467240147
device firmware: 4.0.3916.0
[...]
[Info] [OpenGLDepthPacketProcessor] avg. time: 3.60516ms -> ~277.38Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.97567ms -> ~200.978Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 18.6088ms -> ~53.7379Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 5.55246ms -> ~180.1Hz
Other useful information
Hardware
Kinect v2, connected to USB3.1 port
AMD Ryzen 5900X
Nvidia RTX3060Ti
OBS
Version 27.1.3 (linux)
installed using apt from ppa:obsproject/obs-studio
obs-kinect
Artifact taken from the latest build
Directories
tree -L 3 /usr/share/obs/obs-plugins/obs-kinect/
/usr/share/obs/obs-plugins/obs-kinect/
├── alpha_mask.effect
├── color_multiplier.effect
├── gaussian_blur.effect
├── greenscreen_filter.effect
├── locale
│ ├── cs-CZ.ini
│ ├── de-DE.ini
│ ├── en-US.ini
│ ├── fr-FR.ini
│ ├── ko-KR.ini
│ └── pl-PL.ini
├── texture_lerp.effect
└── visibility_mask.effect
tree -L 1 /usr/lib/obs-plugins
/usr/lib/obs-plugins
├── cef_100_percent.pak
├── cef_200_percent.pak
├── cef_extensions.pak
├── cef.pak
├── chrome-sandbox
├── decklink-captions.so
├── decklink-ouput-ui.so
├── devtools_resources.pak
├── frontend-tools.so
├── icudtl.dat
├── image-source.so
├── libcef.so
├── libEGL.so
├── libGLESv2.so
├── libobs-kinectcore.so
├── linux-alsa.so
├── linux-capture.so
├── linux-decklink.so
├── linux-jack.so
├── linux-pulseaudio.so
├── linux-v4l2.so
├── locales
├── obs-browser-page
├── obs-browser.so
├── obs-ffmpeg.so
├── obs-filters.so
├── obs-kinect-azuresdk.so
├── obs-kinect-freenect2.so
├── obs-kinect-freenect.so
├── obs-kinect.so
├── obs-libfdk.so
├── obs-outputs.so
├── obs-transitions.so
├── obs-vst.so
├── obs-x264.so
├── rtmp-services.so
├── snapshot_blob.bin
├── swiftshader
├── text-freetype2.so
├── v8_context_snapshot.bin
└── vlc-video.so
Hi! Thank you for this report.
As I understand it, you provide a custom libfreenect implementation. However, using the CpuDepthPacketProcessor brings latency and lost packets.
There's two libfreenect, the first one (libfreenect) targets Kinect for 360 and the second one (libfreenect2 - the one you're using) targets Kinect for xbox one.
I had to provide a custom implementation of libfreenect to bypass one of its limitation (which is currently being fixed), but the libfreenect2 implementation is the official one. Which is why I'm a bit surprised by this TurboJPEG error.
Could you show me the output of ldd /usr/lib/obs-plugins/obs-kinect-freenect2.so? Maybe you have an older TurboJPEG library it links to.
As for the depth packet processor, you're right. I will add a way to use the OpenGL/OpenCL pipelines.
Hi! obs-kinect-freenect2.so does not seem to link to TurboJPEG:
linux-vdso.so.1 (0x00007ffdcf3ae000)
libobs.so.0 => /lib/libobs.so.0 (0x00007f8659f22000)
libobs-kinectcore.so => /usr/lib/obs-plugins/libobs-kinectcore.so (0x00007f8659f14000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8659f0f000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8659cf6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8659c12000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8659bf8000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f86599ce000)
/lib64/ld-linux-x86-64.so.2 (0x00007f865a16d000)
libgio-2.0.so.0 => /lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007f86597ff000)
libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f86596c1000)
libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f8659697000)
libX11-xcb.so.1 => /lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007f8659692000)
libpulse.so.0 => /lib/x86_64-linux-gnu/libpulse.so.0 (0x00007f865963d000)
libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007f8658236000)
libjansson.so.4 => /lib/x86_64-linux-gnu/libjansson.so.4 (0x00007f8658226000)
libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007f8657f8d000)
libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007f8657cd4000)
libswscale.so.5 => /lib/x86_64-linux-gnu/libswscale.so.5 (0x00007f8657c36000)
libswresample.so.3 => /lib/x86_64-linux-gnu/libswresample.so.3 (0x00007f8657c14000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f8657bf6000)
libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f8657ac0000)
libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f8657a65000)
libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f8657a5f000)
libmount.so.1 => /lib/x86_64-linux-gnu/libmount.so.1 (0x00007f8657a1e000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f86579f0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f86579eb000)
libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f86579e5000)
libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f86579dd000)
libpulsecommon-15.0.so => /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-15.0.so (0x00007f8657956000)
libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007f8657906000)
libvpx.so.6 => /lib/x86_64-linux-gnu/libvpx.so.6 (0x00007f86576ef000)
libwebpmux.so.3 => /lib/x86_64-linux-gnu/libwebpmux.so.3 (0x00007f86576e3000)
libwebp.so.6 => /lib/x86_64-linux-gnu/libwebp.so.6 (0x00007f865767b000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8657650000)
libaribb24.so.0 => /lib/x86_64-linux-gnu/libaribb24.so.0 (0x00007f865743e000)
libdav1d.so.4 => /lib/x86_64-linux-gnu/libdav1d.so.4 (0x00007f8657308000)
libopencore-amrwb.so.0 => /lib/x86_64-linux-gnu/libopencore-amrwb.so.0 (0x00007f86572f2000)
librsvg-2.so.2 => /lib/x86_64-linux-gnu/librsvg-2.so.2 (0x00007f86568b6000)
libcairo.so.2 => /lib/x86_64-linux-gnu/libcairo.so.2 (0x00007f865678e000)
libzvbi.so.0 => /lib/x86_64-linux-gnu/libzvbi.so.0 (0x00007f86566ff000)
libsnappy.so.1 => /lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007f86566f4000)
libaom.so.0 => /lib/x86_64-linux-gnu/libaom.so.0 (0x00007f8656254000)
libcodec2.so.0.9 => /lib/x86_64-linux-gnu/libcodec2.so.0.9 (0x00007f8655470000)
libgsm.so.1 => /lib/x86_64-linux-gnu/libgsm.so.1 (0x00007f8655461000)
libmp3lame.so.0 => /lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007f86553ec000)
libopencore-amrnb.so.0 => /lib/x86_64-linux-gnu/libopencore-amrnb.so.0 (0x00007f86553c1000)
libopenjp2.so.7 => /lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007f865536b000)
libopus.so.0 => /lib/x86_64-linux-gnu/libopus.so.0 (0x00007f865530b000)
libshine.so.3 => /lib/x86_64-linux-gnu/libshine.so.3 (0x00007f8655100000)
libspeex.so.1 => /lib/x86_64-linux-gnu/libspeex.so.1 (0x00007f86550e2000)
libtheoraenc.so.1 => /lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007f86550a5000)
libtheoradec.so.1 => /lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007f8655085000)
libtwolame.so.0 => /lib/x86_64-linux-gnu/libtwolame.so.0 (0x00007f8655060000)
libvo-amrwbenc.so.0 => /lib/x86_64-linux-gnu/libvo-amrwbenc.so.0 (0x00007f8655042000)
libvorbis.so.0 => /lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007f8655014000)
libvorbisenc.so.2 => /lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007f8654f69000)
libx264.so.160 => /lib/x86_64-linux-gnu/libx264.so.160 (0x00007f8654ca9000)
libx265.so.192 => /lib/x86_64-linux-gnu/libx265.so.192 (0x00007f8653d26000)
libxvidcore.so.4 => /lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007f8653c13000)
libva.so.2 => /lib/x86_64-linux-gnu/libva.so.2 (0x00007f8653be2000)
libmfx.so.1 => /lib/x86_64-linux-gnu/libmfx.so.1 (0x00007f8653bd4000)
libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f86539f2000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f86539df000)
libgme.so.0 => /lib/x86_64-linux-gnu/libgme.so.0 (0x00007f865398f000)
libopenmpt.so.0 => /lib/x86_64-linux-gnu/libopenmpt.so.0 (0x00007f86537a0000)
libchromaprint.so.1 => /lib/x86_64-linux-gnu/libchromaprint.so.1 (0x00007f865378a000)
libbluray.so.2 => /lib/x86_64-linux-gnu/libbluray.so.2 (0x00007f8653733000)
libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f865354e000)
librabbitmq.so.4 => /lib/x86_64-linux-gnu/librabbitmq.so.4 (0x00007f8653535000)
libsrt-gnutls.so.1.4 => /lib/x86_64-linux-gnu/libsrt-gnutls.so.1.4 (0x00007f8653489000)
libssh-gcrypt.so.4 => /lib/x86_64-linux-gnu/libssh-gcrypt.so.4 (0x00007f86533fd000)
libzmq.so.5 => /lib/x86_64-linux-gnu/libzmq.so.5 (0x00007f865333d000)
libva-drm.so.2 => /lib/x86_64-linux-gnu/libva-drm.so.2 (0x00007f8653338000)
libva-x11.so.2 => /lib/x86_64-linux-gnu/libva-x11.so.2 (0x00007f8653330000)
libvdpau.so.1 => /lib/x86_64-linux-gnu/libvdpau.so.1 (0x00007f865332a000)
libdrm.so.2 => /lib/x86_64-linux-gnu/libdrm.so.2 (0x00007f8653314000)
libOpenCL.so.1 => /usr/local/cuda/targets/x86_64-linux/lib/libOpenCL.so.1 (0x00007f865310a000)
libsoxr.so.0 => /lib/x86_64-linux-gnu/libsoxr.so.0 (0x00007f86530a3000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f865302d000)
libffi.so.8 => /lib/x86_64-linux-gnu/libffi.so.8 (0x00007f8653021000)
libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f8652fee000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f8652f56000)
libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f8652f3e000)
libsndfile.so.1 => /lib/x86_64-linux-gnu/libsndfile.so.1 (0x00007f8652ebd000)
libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007f8652e05000)
libasyncns.so.0 => /lib/x86_64-linux-gnu/libasyncns.so.0 (0x00007f8652dfd000)
libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007f8652df1000)
libapparmor.so.1 => /lib/x86_64-linux-gnu/libapparmor.so.1 (0x00007f8652dda000)
libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f8652d9f000)
libcairo-gobject.so.2 => /lib/x86_64-linux-gnu/libcairo-gobject.so.2 (0x00007f8652d93000)
libgdk_pixbuf-2.0.so.0 => /lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007f8652d63000)
libpangocairo-1.0.so.0 => /lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007f8652d51000)
libpango-1.0.so.0 => /lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007f8652cfb000)
libpixman-1.so.0 => /lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007f8652c51000)
libfontconfig.so.1 => /lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f8652c09000)
libfreetype.so.6 => /lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f8652b46000)
libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007f8652b41000)
libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007f8652b32000)
libXrender.so.1 => /lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f8652b23000)
libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007f8652b0e000)
libogg.so.0 => /lib/x86_64-linux-gnu/libogg.so.0 (0x00007f8652b03000)
libnuma.so.1 => /lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f8652af6000)
libicuuc.so.67 => /lib/x86_64-linux-gnu/libicuuc.so.67 (0x00007f865290a000)
libmpg123.so.0 => /lib/x86_64-linux-gnu/libmpg123.so.0 (0x00007f86528ae000)
libvorbisfile.so.3 => /lib/x86_64-linux-gnu/libvorbisfile.so.3 (0x00007f86528a3000)
libudfread.so.0 => /lib/x86_64-linux-gnu/libudfread.so.0 (0x00007f8652898000)
libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f865275e000)
libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f865273b000)
libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f86525b9000)
libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f86525a3000)
libnettle.so.8 => /lib/x86_64-linux-gnu/libnettle.so.8 (0x00007f865255d000)
libhogweed.so.6 => /lib/x86_64-linux-gnu/libhogweed.so.6 (0x00007f8652515000)
libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f8652492000)
libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f86523f8000)
libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f865211d000)
libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f8651ffa000)
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f8651fd2000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f8651f80000)
libsodium.so.23 => /lib/x86_64-linux-gnu/libsodium.so.23 (0x00007f8651f28000)
libpgm-5.3.so.0 => /lib/x86_64-linux-gnu/libpgm-5.3.so.0 (0x00007f8651ed5000)
libnorm.so.1 => /lib/x86_64-linux-gnu/libnorm.so.1 (0x00007f8651daa000)
libXfixes.so.3 => /lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007f8651da2000)
libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f8651d5b000)
libmd.so.0 => /lib/x86_64-linux-gnu/libmd.so.0 (0x00007f8651d4e000)
libFLAC.so.8 => /lib/x86_64-linux-gnu/libFLAC.so.8 (0x00007f8651d0d000)
libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f8651c40000)
liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007f8651c20000)
libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007f8651c15000)
libnsl.so.2 => /lib/x86_64-linux-gnu/libnsl.so.2 (0x00007f8651bfb000)
libjpeg.so.8 => /lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f8651b86000)
libpangoft2-1.0.so.0 => /lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007f8651b6c000)
libfribidi.so.0 => /lib/x86_64-linux-gnu/libfribidi.so.0 (0x00007f8651b50000)
libthai.so.0 => /lib/x86_64-linux-gnu/libthai.so.0 (0x00007f8651b45000)
libharfbuzz.so.0 => /lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007f8651a5d000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f8651a2b000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f8651a22000)
libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007f8651a14000)
libicudata.so.67 => /lib/x86_64-linux-gnu/libicudata.so.67 (0x00007f864fefb000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f864fe32000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f864fe03000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f864fdfd000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f864fdef000)
libtirpc.so.3 => /lib/x86_64-linux-gnu/libtirpc.so.3 (0x00007f864fdc2000)
libdatrie.so.1 => /lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007f864fdb8000)
libgraphite2.so.3 => /lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007f864fd89000)
libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007f864fd66000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f864fd5f000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f864fd4a000)
However, if I run ldd on build/lib/libfreenect2.so (a local build of libfreenect2), this is the output:
linux-vdso.so.1 (0x00007fff365fd000)
libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007fccd6446000)
libva-drm.so.2 => /lib/x86_64-linux-gnu/libva-drm.so.2 (0x00007fccd6441000)
libva.so.2 => /lib/x86_64-linux-gnu/libva.so.2 (0x00007fccd6412000)
libjpeg.so.8 => /lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007fccd639f000)
libturbojpeg.so.0 => /lib/x86_64-linux-gnu/libturbojpeg.so.0 (0x00007fccd6315000)
libglfw.so.3 => /lib/x86_64-linux-gnu/libglfw.so.3 (0x00007fccd62ac000)
libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007fccd6223000)
libOpenCL.so.1 => /usr/local/cuda/targets/x86_64-linux/lib/libOpenCL.so.1 (0x00007fccd601b000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fccd5e02000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fccd5d1e000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fccd5af6000)
/lib64/ld-linux-x86-64.so.2 (0x00007fccd65b0000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fccd5acd000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fccd5ac6000)
libdrm.so.2 => /lib/x86_64-linux-gnu/libdrm.so.2 (0x00007fccd5ab0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fccd5aab000)
libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007fccd596d000)
libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007fccd58b5000)
libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007fccd587f000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fccd5865000)
libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fccd583b000)
libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007fccd5835000)
libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fccd582d000)
libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fccd5813000)
libmd.so.0 => /lib/x86_64-linux-gnu/libmd.so.0 (0x00007fccd5806000)
This is because libfreenect2 was compiled as a static library, please check the last nightly, it links libfreenect2 dynamically (but you'll need it in your /usr/lib/x86_64-linux-gnu folder)
I confirm it works now using latest nightly, with libfreenect2 libraries in /usr/lib/x86_64-linux-gnu. Thanks a lot, @SirLynix!
Glad to hear it works! I'll try to improve it and enable GPU depth processing :)
I've put the files in what I think are the right locations, and obs has the plugin loaded, but it doesn't see the device.
I have attached the log and a screenshot of obs.
I'd really like to know where exactly the built files go. It seems like they're still not in the right place, and building them doesn't put them in the right place. A tree with the correct file placements would be really helpful!
The my KinectV1 is connected correctly, as I can see the video source as a V4L2 video capture device, but my use case is for the depth sensor and microphone.
Log.txt
From your logs i think you did not install libfreenect in the /usr/lib/obs-plugins. Clone it from SirLynix repository and then run a make install. Next you need to symlink it to the obs plugin directory.
Indeed this is the issue:
error: os_dlopen(./obs-kinect-freenect->./obs-kinect-freenect.so): libfreenect.so.0: cannot open shared object file: No such file or directory
I would like to make a helpful guide saying "hey just put those files there" but since I don't use Linux at all I'm not sure which folder would be the best (or how I should distribute obs-kinect to work on most distributions).
From your logs i think you did not install libfreenect in the /usr/lib/obs-plugins. Clone it from SirLynix repository and then run a make install. Next you need to symlink it to the obs plugin directory.
Hey so I'm trying this again and I get the same exact issue. Where is the OBS plugins directory that I need to link to? I've found 4 separate obs-plugins folders so far, and none of them seem to be the correct one.
I have even tried building obs and then linking it to that directory, but still no luck.
SirLynix, start with debian. A lot of people use debian. Once it works on debian, someone will likely make a pull for the other distros.
I've finally figured out how to install it in flatpak. This is definitely not how you're supposed to do things in flatpak (and will probably be erased after every obs update), but it's better than nothing. This works no matter what distro you're on. So if you're struggling to install it, use the flatpak. I'll also look into flatpak extensions as this seems to be the proper way of doing it.
- Install the official OBS flatpak (https://obsproject.com/download)
- If you haven't already, copy the libfreenect2 udev rules to
/etc/udev/rules.d/(restart after that) - Make a directory somewhere. I'll make
~/obs-kinect - compile the extension and copy
ubuntu-latest-x64-releasedbg/obs-plugins/64bit/*files to obs-plugins in obs flatpak and your newly created directory. If installed globally (default), copy it to/var/lib/flatpak/app/com.obsproject.Studio/current/active/files/lib/obs-plugins/AND to~/obs-kinect - copy
ubuntu-latest-x64-releasedbg/data/obs-plugins/obs-kinect/folder to/var/lib/flatpak/app/com.obsproject.Studio/current/active/files/share/obs/obs-plugins - compile libfreenect2 and put
libfreenect2.so,libfreenect2.so.0.2, andlibfreenect2.so.0.2.0in~/obs-kinect(I've forked the repo with an action that builds it if you can't build it) - You'll also need the libusb and libglfw libraries as they aren't provided by the obs flatpak. They should be already in your system, but in my case they're compiled against a higher glibc version than what is in the flatpak. So I downloaded the ubuntu 20.04 "focal" packages for
libglfw3andlibusb-1.0-0from https://packages.ubuntu.com/ and extractedlibglfw.so.3,libglfw.so.3.3,libusb-1.0.so.0, andlibusb-1.0.so.0.2.0to~/obs-kinect - Now override the obs flatpak filesystem to add
/home/<your_username>/obs-kinect:roalso override the environment variables to addLD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/<your_username>/obs-kinectIf obs crashes for you, also addLIBFREENECT2_PIPELINE=cpu(see comment below). I used flatseal for all of this, but you can useflatpak override --usercommand if you like.
After that I get to the same point where I got with the "native" installation. It works but crashes 90% of the time.
@SirLynix if you're still working on this project, this is the output of when it crashes: crashed.txt
and this is it when it launches properly: launched.txt (though I think it crashed when I tried to exit there)
See #9 (comment)
it looks like a conflict between freenect2 OpenGL usage with obs-studio OpenGL usage.
Could you try to run set LIBFREENECT2_PIPELINE=cpu before obs to see if it fixes those issues?
Thanks a lot for your investigations by the way 😄
That was it! It now runs reliably. I'll now look through flatpak documentation for extensions.
I'm glad it fixed it!
The downside is that it will probably have a higher CPU cost, but I'm not sure how much since this plugin is already a bit heavy on the CPU.
I'm not sure how I can fix this while keeping the OpenGL decoding, probably by threading the whole image processing.
I'm not sure what I'm doing wrong here. I've installed the udev rules, compiled and installed libfreenect and libfreenect2, copied the libraries to the instructed places, etc.
I do not have a usb3.0 port, but that shouldn't matter... My kinect is a USB2.0 device.
This is what shows up in OBS:
There are no devices in that dropdown list.
Using the kinect as a webcam works fine.
I do not need the LIBFREENECT2_PIPELINE=cpu line yet, but that might just be because it's not detecting my kinect.
I'm happy that this project is in the final stages of development :D
Could you share your obs logs?
Sure, where do I find those again?
Check help => log files => show log files/view current log.
Thank you.
Your issue appears clearly in the logs, obs-kinect fails to find libjpeg.so.8:
18:37:53.186: os_dlopen(/app//lib/obs-plugins/obs-kinect-freenect2.so->/app//lib/obs-plugins/obs-kinect-freenect2.so): libjpeg.so.8: cannot open shared object file: No such file or directory
18:37:53.186:
18:37:53.192: os_dlopen(/app//lib/obs-plugins/obs-kinect-freenect2.so->/app//lib/obs-plugins/obs-kinect-freenect2.so): libjpeg.so.8: cannot open shared object file: No such file or directory
18:37:53.192:
18:37:53.192: Module '/app//lib/obs-plugins/obs-kinect-freenect2.so' not loaded
Could you try to install libjpeg-turbo? (libjpeg-turbo8 package in apt)
It is already installed.
I'll try adding it manually to ~/obs-kinect
2022-12-28 18-47-10.txt
It looks like the obs-kinect build is not compatible with your system (what's your distribution?):
/usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /home/user/obs-kinect/libfreenect2.so.0.2)
I recommend you to try to build it yourself:
- build obs-studio
- write a config.lua to tell where it's located
- get xmake: https://xmake.io/#/guide/installation
- clone this repository
- build using
xmake - copy all relevant files to a folder using
xmake install -o install_package
You can check the CI build script as a reference: https://github.com/SirLynix/obs-kinect/blob/master/.github/workflows/linux-build.yml#L131
Sorry you have to go through this, this is why I find it's a pain in the ass to distribute an app (or even a plugin) on Linux systems.
I think the problem now is knowing where all of the files in the plugin go. I can build everything just fine, but the naming schema of both obs and the plugins are all a bit samey.
I am using Linux mint 21 currently, which is based on ubuntu. I'm going to build the plugin and see if a newer version of glibc fixes things.
Snap might be a solution...
Have you tried building this recently on linux? It's not building. Specifically, k4a is not building. Are you using docker to do your builds?
Can you please rebuild this with the latest glibc? Building from source is completely broken. I'm either missing arbitrary dependencies, or xmake is screwing stuff up. You have a working toolchain.
The current stable release of libc across most debian-based distros is 2.35. The latest release for bionic (the one that you're building for) is 2.27.
I don't even need libfreenect2. I just need freenect. I have an Xbox360 kinect 1473. It saddens me that this project, while on the brink of completion, has not made much progress in the past few years. Making this a portable library shouldn't be this difficult...
Making this a portable library shouldn't be this difficult...
That's the main issue with linux distribution, you need instructions for almost every distribution family and distribution version, or you rely on package managers (which I doubt is possible for an obs plugin). and there's stuff like snap which complicate this thing even more.
The library itself is portable (it should even compile and work fine on macOS), it's its distribution / installation which is not.
The current stable release of libc across most debian-based distros is 2.35. The latest release for bionic (the one that you're building for) is 2.27.
Have you tried building this recently on linux? It's not building. Specifically, k4a is not building. Are you using docker to do your builds?
Linux binaries are built using Ubuntu 18.04 which should be fine, it's probably libfreenect that has unmet requirements, try ldd on your freenect so.
Also you're right, you don't need freenect2, only freenect.
18:47:12.253: os_dlopen(/app//lib/obs-plugins/obs-kinect-freenect.so->/app//lib/obs-plugins/obs-kinect-freenect.so): libfreenect.so.0: cannot open shared object file: No such file or directory
18:47:12.253:
18:47:12.265: os_dlopen(/app//lib/obs-plugins/obs-kinect-freenect.so->/app//lib/obs-plugins/obs-kinect-freenect.so): libfreenect.so.0: cannot open shared object file: No such file or directory
18:47:12.265:
18:47:12.265: Module '/app//lib/obs-plugins/obs-kinect-freenect.so' not loaded
it looks like it just can't find it, check the file is there and try ldd obs-kinect-freenect.so.
@weirdal3333 Sorry about that, I've made this with only libfreenect2 in mind. So xbox 360 kinect and azure kinect probably have different requirements.
You'll have to compile libfreenect, but try downloading it first from the artifacts of this github action and place the libfreenect .so libraries in ~/obs-kinect Also, to eliminate as much variables with glibc as we can, download obs-kinect from SirLynix's action (I also see in your screenshot that old obs-kinect versions separate some libs in /bin/64bit/* so just use this latest build).
If you still have problems with glibc versions, download precompiled libraries from ubuntu 20.04 debs. For example, libglfw3, libusb-1.0-0, and libturbojpeg. And extract .so libraries from them to ~/obs-kinect
I don't have an xbox 360 kinect test with. But, looking at the logs, I think those were all the libraries it needed.
@SirLynix I've finally built a (somewhat) working flatpak! I also found out that if you build libfreenect2 without glfw, it doesn't crash anymore! You can try it out right now:
flatpak install https://dl.flathub.org/build-repo/127309/com.obsproject.Studio.Plugin.obs-kinect.flatpakref
The only problem is it can't find obs-kinect-freenect2.so in the default obs-plugins folder for some reason, so for now, you'll still have to set LD_LIBRARY_PATH but this time to /app/plugins/lib/obs-plugins
@weirdal3333 It now builds with libfreenect for kinect v1, but I don't have that kinect to test with (I think I need to build glut and turbojpeg). So could you please try installing that build and tell me how it goes? But before installing you should reinstall obs to remove the plugins we added manually. After that set LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/app/plugins/lib/obs-plugins in flatseal (you can also remove the filesystem perm for ~/obs-kinect and delete that folder).
Thanks a lot for your efforts!
The only problem is it can't find obs-kinect-freenect2.so in the default obs-plugins folder for some reason, so for now, you'll still have to set LD_LIBRARY_PATH but this time to /app/plugins/lib/obs-plugins
I wonder how this could be solved, I'm already setting the rpath when building the lib 🤔
@SirLynix Could you please make a new stable release with the latest ubuntu ci? It's flathub's policy to use stable release tags and not development snapshots for apps. And also... compiling obs-kinect and libobs doesn't look like an easy task, so I'll just use the binaries from the ubuntu ci for now 😅
As for what works, kinect v2 does, kinect v1 should work though I can't test it, and finally azure kinect won't work yet. Looks like libk4a has a more involved build process, but what concerns me the most is this depth sensor dependency (does obs-kinect even use it?)
Anyways, I'll take care of the rest later. For now, I just want to get it on flathub and (hopefully) change your mind about how difficult it is to distribute apps on GNU/Linux :)
Thanks for all your work!
Yeah obs-kinect uses the closed-source depth sensor dependency with the Azure Kinect, there's no way around this afaik.
I released a new version which should work properly this time.
Well, it still doesn't work.