Hand tracking mostly doesn't work for fingers

Question

Hand tracking mostly doesn't work for fingers

Anicrof opened this issue 4 years ago · comments

Hi, i'm having an issue using Hand-tracking in unity through Oculus Link to setup poses.

Everything works perfectly when using controllers but so far, I've managed to have hand-tracking working only on 2 tests out of a lot.

Generally, my hands remains in "default pose" even though they actually follow my movements ; but the bones don't move at all, and the hand is in a weird rotation. I can still grab the objects and the hand will take the right pose, but as soon as I let go, they go back to there default position.
Also when it is working, I see the skeleton on my hand, but it is not rendered when not working

I managed to record poses the 2 times it was working tho, I just want it to work everytime.
Works fine in android builds also.

I'm using Unity 2019.3.15f1, the last version of oculus intergration, XR interaction Toolkit 0.9.4 and Oculus XR plugin 1.3.4

Luca Mefisto · Answer 1 · Wed Jul 22 2020 21:46:45 GMT+0800 (China Standard Time)

Are you talking about using it in the Unity Engine directly or in a PC build? Also this is using the Oculus Integration 18.0 from the asset store/ Oculus build, not necesarily XR interaction toolkit yet. If you want to try with XRinteraction toolkit instead, try changing the Tracked Hand Offset as specified in: https://github.com/MephestoKhaan/HandPosing/wiki/Hand-Puppeting

Anicrof · Answer 2 · Wed Jul 22 2020 22:33:52 GMT+0800 (China Standard Time)

I tried creating an empty project and downloading the latest version of Oculus Integration (18.0).
To answer your first question, i'm trying to work in Unity from the Oculus link in order to record poses in the sample scene.

I uninstalled the Oculus XR Plugin, but I require XR Inteeraction Toolkit in order to run the Unity App in the quest via Oculus link : if I don't have it, I can only see Unity as a virtual panel when I press play, but not actually control the player in the game view.

Now it still works in apk build and in the Editor with controller but switching to hands is even worse than before; they don't even follow my moves anymore

Can you tell me exactly which packages you are using ?
Or any other idea ?

Thanks !

Luca Mefisto · Answer 3 · Wed Jul 22 2020 22:38:00 GMT+0800 (China Standard Time)

I literally just create a new project in Unity 2019.3, import the asset store Oculus Integration (18.0), bring in the package from here: https://github.com/MephestoKhaan/HandPosing/releases and it all worked.
I did not do anything special besides that, not even the XR interaction toolkit and it works with hand tracking with link in the editor.

Give it a try and thanks for your detailed explanation! I have a busy week ahead but will try to give it a spin tomorrow/friday and get back to you.

Anicrof · Answer 4 · Wed Jul 22 2020 23:27:49 GMT+0800 (China Standard Time)

I started from point zero again, created a project, downlaoded the Oculus integration 18.0, and used the unity package you provided.
I did not say in my previous posts, but I do have compiler errors. I actually just fixed them in the previous tests, maybe that's where the probleme come from :

Basically to fix those, I just created properties that are called IsInitialized and IsDataValid and made them return _isInitialized and data.IsDataValid. I think that would have done the trick ? Don't you have those errors ?

I also have to set hand tracking support in the OVRManager :

Then in the project Settings I must add Oculus to the XR Settings :

Then removing Vulkan if needed, and setting the right target API but all of this is for building.

Oculus link doesn't put me in the game in this case 😢
I don't know where we are doing things differently. Hope you can find out with me.

Good luck on your work, i'll update if I find something

Update : To ensure I can work in play mode with oculus link, the package needed is Oculus Desktop. But still hand-tracking won't work at all

Anicrof · Answer 5 · Thu Jul 23 2020 00:02:27 GMT+0800 (China Standard Time)

Ok I found out !

I don't know why but my Oculus Integration wasn't updating even though I downloaded it again and imported it in each test project : the reason to this was that I have been unintentionnaly disconnected from my account at some point.
When reconnecting, Unity told me that I could upgrade Oculus Integration if I wanted and I was like "Yeah, I just did that", but did it anyway, and everything worked perfectly, I'm speechless.

Sorry for taking your time, thanks for the help, and thanks for this incredible asset, this is remarkable.
I'll let you close this issue, good luck on what's next :)

Luca Mefisto · Answer 6 · Thu Jul 23 2020 00:10:30 GMT+0800 (China Standard Time)

Oh neat!
I am very glad it worked. I know Oculus plugin can be a bit of a pain to update properly etc...

I will close this, but thanks for bringing my attention to the Unity XR integration, watch this issue: #22 as I will put there my progress when time permits.