Cannot control hands on Quest 2 for HandGamePlayShowcase
wetoo-cando opened this issue · comments
Hi,
I cloned the Oculus fork of UE from here: https://github.com/Oculus-VR/UnrealEngine
Checked out branch oculus-5.3.
Also cloned the main branch of: https://github.com/oculus-samples/Unreal-HandGameplay
Built the Engine and the Unreal-HandGameplay project using Visual Studio 2022.
Then I ran the HandGamePlayShowcase level in VR mode.
I can see the scene in the headset, however I cannot control the hands.
What could be wrong?
Could you verify that hand tracking is enabled in your headset? Here are instructions on how to do that.
If that doesn't fix it, could you please provide us with the contents of the Output Log in your Unreal Editor? Thanks.
Thanks for the reponse!
The log is attached below. It shows:
OpenXR runtime name: Oculus, version 1.94.0
OpenXR hand tracking system properties: hand tracking supported 1
Hand tracking is definitely enabled in Quest 2 settings and I can also use it on the Quest native interface. However when I enable Quest link, on the screen with the dashboard I need to use the controllers. Hands don't work. From the dashboard, when I go into desktop mode (where I see the Windows desktop) again hands don't work, only controllers.
Not sure what I should about all the Unavailable OpenXR extensions. May be this is the issue?
Also, this looks suspicious: Warning: Hand Tracking: Unable to find bone named 'b_r_thumb_ignore' in hand skeleton.
LogDebuggerCommands: Repeating last play command: VR Preview
LogPlayLevel: PlayLevel: No blueprints needed recompiling
LogHMD: Initializing OVRPlugin session
LogOVRPlugin: Display: OpenXR SDK version 1.0.30
LogOVRPlugin: Display: commandQueue is null
LogOVRPlugin: CompositorOpenXR::Initialize()
LogOVRPlugin: CompositorOpenXR::Initialize() - refresh available extensions
LogOVRPlugin: Display: Available Extensions: (37)
LogOVRPlugin: Display: Name=XR_EXT_hand_tracking SpecVersion=4
LogOVRPlugin: Display: Name=XR_EXT_hand_joints_motion_range SpecVersion=1
LogOVRPlugin: Display: Name=XR_EXT_hand_tracking_data_source SpecVersion=1
LogOVRPlugin: Display: Name=XR_MSFT_hand_interaction SpecVersion=1
LogOVRPlugin: Display: Name=XR_FB_hand_tracking_aim SpecVersion=2
LogOVRPlugin: Display: Name=XR_ULTRALEAP_hand_tracking_forearm SpecVersion=1
LogOVRPlugin: Display: Name=XR_KHR_D3D11_enable SpecVersion=9
LogOVRPlugin: Display: Name=XR_KHR_D3D12_enable SpecVersion=9
LogOVRPlugin: Display: Name=XR_KHR_opengl_enable SpecVersion=10
LogOVRPlugin: Display: Name=XR_KHR_vulkan_enable SpecVersion=8
LogOVRPlugin: Display: Name=XR_KHR_vulkan_enable2 SpecVersion=2
LogOVRPlugin: Display: Name=XR_KHR_composition_layer_depth SpecVersion=6
LogOVRPlugin: Display: Name=XR_KHR_win32_convert_performance_counter_time SpecVersion=1
LogOVRPlugin: Display: Name=XR_KHR_convert_timespec_time SpecVersion=1
LogOVRPlugin: Display: Name=XR_KHR_composition_layer_cube SpecVersion=8
LogOVRPlugin: Display: Name=XR_KHR_composition_layer_cylinder SpecVersion=4
LogOVRPlugin: Display: Name=XR_KHR_composition_layer_equirect SpecVersion=3
LogOVRPlugin: Display: Name=XR_KHR_visibility_mask SpecVersion=2
LogOVRPlugin: Display: Name=XR_KHR_composition_layer_color_scale_bias SpecVersion=5
LogOVRPlugin: Display: Name=XR_EXT_win32_appcontainer_compatible SpecVersion=1
LogOVRPlugin: Display: Name=XR_EXT_debug_utils SpecVersion=5
LogOVRPlugin: Display: Name=XR_OCULUS_recenter_event SpecVersion=1
LogOVRPlugin: Display: Name=XR_OCULUS_audio_device_guid SpecVersion=1
LogOVRPlugin: Display: Name=XR_FB_color_space SpecVersion=3
LogOVRPlugin: Display: Name=XR_FB_display_refresh_rate SpecVersion=1
LogOVRPlugin: Display: Name=XR_META_performance_metrics SpecVersion=2
LogOVRPlugin: Display: Name=XR_META_headset_id SpecVersion=1
LogOVRPlugin: Display: Name=XR_OCULUS_ovrsession_handle SpecVersion=1
LogOVRPlugin: Display: Name=XR_EXT_local_floor SpecVersion=1
LogOVRPlugin: Display: Name=XR_EXT_active_action_set_priority SpecVersion=1
LogOVRPlugin: Display: Name=XR_FB_haptic_amplitude_envelope SpecVersion=1
LogOVRPlugin: Display: Name=XR_FB_haptic_pcm SpecVersion=1
LogOVRPlugin: Display: Name=XR_FB_touch_controller_pro SpecVersion=1
LogOVRPlugin: Display: Name=XR_META_touch_controller_plus SpecVersion=1
LogOVRPlugin: Display: Name=XR_FB_touch_controller_proximity SpecVersion=1
LogOVRPlugin: Display: Name=XR_OCULUS_external_camera SpecVersion=1
LogOVRPlugin: Display: Name=XR_META_detached_controllers SpecVersion=1
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_EXT_performance_settings
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_hand_tracking_mesh
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_hand_tracking_capsules
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_swapchain_update_state
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_swapchain_update_state_vulkan
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_foveation
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_foveation_configuration
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_foveation_vulkan
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_vulkan_swapchain_create_info
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_foveation_eye_tracked
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_composition_layer_image_layout
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_KHR_composition_layer_equirect2
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_passthrough
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_triangle_mesh
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_passthrough_keyboard_hands
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_passthrough_color_lut
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_passthrough_preferences
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_passthrough_layer_ready_event
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_render_model
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_keyboard_tracking
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_virtual_keyboard
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_scene
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_spatial_entity
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_spatial_entity_container
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_spatial_entity_query
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_spatial_entity_storage
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_spatial_entity_storage_batch
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_spatial_entity_sharing
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_spatial_entity_user
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_spatial_entity_mesh
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_face_tracking
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_face_tracking2
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_eye_tracking_social
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_FB_body_tracking
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_body_tracking_full_body
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_body_tracking_fidelity
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_body_tracking_calibration
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_METAX2_detached_controllers
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_METAX1_simultaneous_hands_controllers_management
LogOVRPlugin: Display: Unavailable OpenXR extension: XR_META_environment_depth
LogOVRPlugin: Display: Initialize: xrCreateInstance() succeeded
LogOVRPlugin: Display: OpenXR runtime name: Oculus, version 1.94.0
LogOVRPlugin: Display: xrGetSystem() succeeded. xrSystemId 20
LogOVRPlugin: Display: OpenXR system properties:
venderId 4294955582, systemName Oculus Quest2, maxSwapchainImage(4096, 4096), maxLayerCount 16, orientationTracking 1, positionTracking 1
LogOVRPlugin: Display: OpenXR hand tracking system properties:
hand tracking supported 1
LogOVRPlugin: Display: OpenXR headset id system properties:
id 525078A55AAB4DC9B8021EBD6C7CFFF4
LogOVRPlugin: Expected XrViewConfigurationType found (XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO)
LogOVRPlugin: XrViewConfigurationProperties: type XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO, fovMutable 0
LogOVRPlugin: Display: xrEnumerateViewConfigurationViews() found 2 views
LogOVRPlugin: Display: View: Recommended Width=2080 Height=2096 SampleCount=1 Maximum Width=4160 Height=4192 SampleCount=4
LogOVRPlugin: Display: View: Recommended Width=2080 Height=2096 SampleCount=1 Maximum Width=4160 Height=4192 SampleCount=4
LogOVRPlugin: Display: Supported Performance Metrics: /perfmetrics_meta/app/cpu_frametime
LogOVRPlugin: Display: Supported Performance Metrics: /perfmetrics_meta/app/gpu_frametime
LogOVRPlugin: Display: Supported Performance Metrics: /perfmetrics_meta/compositor/cpu_frametime
LogOVRPlugin: Display: Supported Performance Metrics: /perfmetrics_meta/compositor/gpu_frametime
LogOVRPlugin: Display: Supported Performance Metrics: /perfmetrics_meta/compositor/dropped_frame_count
LogOVRPlugin: Display: Supported Performance Metrics: /perfmetrics_meta/compositor/spacewarp_mode
LogOVRPlugin: Display: CompositorOpenXR::Initialize() - succeeded
LogOVRPlugin: Display: OVRPlugin 1.92.0 ... initialized
LogOVRPlugin: HandleOpenXREvents(): XrEventDataInteractionProfileChanged
LogOVRPlugin: Display: [XR_SESSION] XrEventDataSessionStateChanged: state XR_SESSION_STATE_UNKNOWN->XR_SESSION_STATE_IDLE session=72180206764294145 time=873316915058300
LogOVRPlugin: [XR_SESSION] ... running 0 visible 0 focused 0
LogOVRPlugin: Display: [XR_SESSION] XrEventDataSessionStateChanged: state XR_SESSION_STATE_IDLE->XR_SESSION_STATE_READY session=72180206764294145 time=873316915121699
LogOVRPlugin: [XR_SESSION] ... running 1 visible 0 focused 0
LogOVRPlugin: CompositorOpenXR_D3D12::Layer::Initialize(): CreateSwapchain for eye 0: 04006F8400000001, 3 stages
LogOVRPlugin: ... m_textureStages[0][0] = 0000016032247790
LogOVRPlugin: ... m_textureStages[0][1] = 0000016032248A20
LogOVRPlugin: ... m_textureStages[0][2] = 0000016032248600
LogOVRPlugin: CompositorOpenXR_D3D12::Layer::CreateDepthSwapChain(): [depthStencil] CreateSwapchain for eye 0: 04006F8400000002, 3 stages
LogOVRPlugin: ... m_depthTextureStages[0][0] = 00000160356BF3B0
LogOVRPlugin: ... m_depthTextureStages[0][1] = 00000160356BE750
LogOVRPlugin: ... m_depthTextureStages[0][2] = 00000160356BF9E0
LogHMD: Allocating Oculus 4160 x 2096 rendertarget swapchain
LogOVRPlugin: Display: [CompositorOpenXR::SetClientColorDesc] Change colorspace from 0 to 7
PIE: New page: PIE session: HandGameplayShowcase (16 Feb 2024, 17:36:07)
LogPlayLevel: Creating play world package: /Game/HandGameplay/Levels/UEDPIE_0_HandGameplayShowcase
LogPlayLevel: PIE: StaticDuplicateObject took: (0.009029s)
LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/HandGameplay/Levels/HandGameplayShowcase.HandGameplayShowcase to /Game/HandGameplay/Levels/UEDPIE_0_HandGameplayShowcase.HandGameplayShowcase (0.009319s)
LogUObjectHash: Compacting FUObjectHashTables data took 0.41ms
LogWorldSubsystemInput: UEnhancedInputDeveloperSettings::bEnableWorldSubsystem is false, the world subsystem will not be created!
LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000
LogAIModule: Creating AISystem for world HandGameplayShowcase
LogPlayLevel: PIE: World Init took: (0.001430s)
LogAudio: Display: Creating Audio Device: Id: 11, Scope: Unique, Realtime: True
LogAudioMixer: Display: Audio Mixer Platform Settings:
LogAudioMixer: Display: Sample Rate: 48000
LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024
LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024
LogAudioMixer: Display: Number of buffers to queue: 2
LogAudioMixer: Display: Max Channels (voices): 32
LogAudioMixer: Display: Number of Async Source Workers: 0
LogAudio: Display: AudioDevice MaxSources: 32
LogAudio: Display: Audio Spatialization Plugin: None (built-in).
LogAudio: Display: Audio Reverb Plugin: None (built-in).
LogAudio: Display: Audio Occlusion Plugin: None (built-in).
LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2'
LogAudioMixer: Display: Using Audio Hardware Device Headphones (Oculus Virtual Audio Device)
LogAudioMixer: Display: Initializing Sound Submixes...
LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault'
LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault'
LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=11
LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=11
LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=11
LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=11
LogInit: FAudioDevice initialized with ID 11.
LogAudio: Display: Audio Device (ID: 11) registered with world 'HandGameplayShowcase'.
LogAudioMixer: Initializing Audio Bus Subsystem for audio device with ID 11
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogViewport: Scene viewport resized to 4160x2096, mode Windowed.
LogLoad: Game class is 'HandsGameMode_C'
LogWorld: Bringing World /Game/HandGameplay/Levels/UEDPIE_0_HandGameplayShowcase.HandGameplayShowcase up for play (max tick rate 3) at 2024.02.16-16.36.07
LogWorld: Bringing up level for play took: 0.004704
LogOnline: OSS: Created online subsystem instance for: :Context_11
LogHandInput: Warning: Hand Tracking: Unable to find bone named 'b_r_thumb_ignore' in hand skeleton.
LogHandInput: Warning: Hand Tracking: Unable to find bone named 'b_r_thumb_ignore' in hand skeleton.
LogHandInput: Warning: Hand Tracking: Unable to find bone named 'b_r_thumb_ignore' in hand skeleton.
LogHandInput: Warning: Hand Tracking: Unable to find bone named 'b_r_thumb_ignore' in hand skeleton.
LogHandInput: Warning: Hand Tracking: Unable to find bone named 'b_l_thumb_ignore' in hand skeleton.
LogHandInput: Warning: Hand Tracking: Unable to find bone named 'b_r_thumb_ignore' in hand skeleton.
LogHandInput: Warning: Hand Tracking: Unable to find bone named 'b_l_thumb_ignore' in hand skeleton.
PIE: Server logged in
PIE: Play in editor total start time 73,957 seconds.
LogOVRPlugin: Display: [XR_SESSION] XrEventDataSessionStateChanged: state XR_SESSION_STATE_READY->XR_SESSION_STATE_SYNCHRONIZED session=72180206764294145 time=873390719081700
LogOVRPlugin: [XR_SESSION] ... running 1 visible 0 focused 0
LogOVRPlugin: Display: [XR_SESSION] XrEventDataSessionStateChanged: state XR_SESSION_STATE_SYNCHRONIZED->XR_SESSION_STATE_VISIBLE session=72180206764294145 time=873391712007600
LogOVRPlugin: [XR_SESSION] ... running 1 visible 1 focused 0
LogOVRPlugin: Display: [XR_SESSION] XrEventDataSessionStateChanged: state XR_SESSION_STATE_VISIBLE->XR_SESSION_STATE_FOCUSED session=72180206764294145 time=873391712063000
LogOVRPlugin: [XR_SESSION] ... running 1 visible 1 focused 1
LogViewport: Scene viewport resized to 1280x720, mode Windowed.
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogWorld: BeginTearingDown for /Game/HandGameplay/Levels/UEDPIE_0_HandGameplayShowcase
LogSlate: Window 'HandGameplay Preview [NetMode: Standalone 0] (64-bit/PC D3D SM5) OculusXRHMD OVRPlugin: 1.92.0' being destroyed
LogWorld: UWorld::CleanupWorld for HandGameplayShowcase, bSessionEnded=true, bCleanupResources=true
LogSlate: InvalidateAllWidgets triggered. All widgets were invalidated
LogPlayLevel: Display: Shutting down PIE online subsystems
LogSlate: InvalidateAllWidgets triggered. All widgets were invalidated
LogAudioMixer: Deinitializing Audio Bus Subsystem for audio device with ID 11
LogAudioMixer: FMixerPlatformXAudio2::StopAudioStream() called. InstanceID=11
LogAudioMixer: FMixerPlatformXAudio2::StopAudioStream() called. InstanceID=11
LogUObjectHash: Compacting FUObjectHashTables data took 0.35ms
LogPlayLevel: Display: Destroying online subsystem :Context_11
LogHMD: Warning: Destroying layer 1
LogOVRPlugin: CompositorOpenXR::~CompositorOpenXR()
LogOVRPlugin: m_xrSession destroyed
LogOVRPlugin: m_xrInstance destroyed
LogUObjectHash: Compacting FUObjectHashTables data took 0.51ms```
Hi,
I’m experiencing the same issue. I’m not sure if hand tracking works with the link mode. Hand tracking relies on the sensors in the headset. However, when using the link mode, the Quest behaves like a Rift. And historically, the Rift has never had hand tracking.
Could you confirm that ?
Thanks.
I'm 100% sure that the following works:
Quest2+UE4.7+Desktop-link
UE4.27 -- Standard epic installation (not oculus fork)
I've verified that myself with https://github.com/3dperceptionlab/unrealhandgrasp
Any ideas @jasonmeisel ?
Any ideas @jasonmeisel ?
Are you still having this issue? We were not able to reproduce it.
I have up after that.
I also had this problem. The fix is to enable Developer Runtime Features in the Oculus/Meta Quest Link app. This let me use my hands in VR preview mode.
i.e. Meta Quest Link App (Not Meta Quest Developer) > Settings > Beta > Developer Runtime Features > Turn everything on.
More here: