Spatial navigation support for AR/VR controllers with D-pads
benzac-de opened this issue Β· comments
Is your feature request related to a problem? Please describe.
TV apps (i.e. web apps designed for large screen devices) look great in a VR environment (like the Wolvic browser). However, most can only be controlled with the navigation keys (i.e. LEFT/RIGHT/UP/DOWN/ENTER/BACK). Currently, these apps cannot be used within the Wolvic browser, because only mouse support seems to be integrated.
Describe the solution you'd like
Is it possible to provide an option to enable spatial navigation by mapping the AR/VR controller D-pad input to LEFT/RIGHT/UP/DOWN key events?
I have already noticed that ENTER and BACK are already mapped to key events on an Oculus Quest 2 controller.
Describe alternatives you've considered
n/a
Additional context
Here is an example TV web app that can be used with mouse and keyboard: http://msx.benzac.de?scale=default&zoom=auto
Although this app can be controlled entirely with mouse, spatial navigation with a controller would be desirable.
For more information about this app, please see this info page: https://msx.benzac.de/info/
We do handle MotionEvents generated from the D-pads and pass it to the web engine (gecko), and I can see it works at least when you want to scroll the web page with D-pads.
And unfortunately, this is something related to the web engine API, I'm not sure if we can do more for it, but let's see.
Thank you for your quick response.
I think a solution approach would be to select in an options panel whether a handleScrollEvent
should dispatch an actual scroll
event or a keyEvent
.
For example, instead of a scroll-up, the following code could be executed (similar to the handleBack
function dispatching a KeyEvent.KEYCODE_BACK
event).
dispatchKeyEvent(new KeyEvent(KeyEvent.ACTION_DOWN, KeyEvent.KEYCODE_DPAD_UP));
dispatchKeyEvent(new KeyEvent(KeyEvent.ACTION_UP, KeyEvent.KEYCODE_DPAD_UP));
I tried to do as you proposed, #1113 But unfortunately that only works for text-editing. I guess we may still need some kind of API to directly pass the keyboard events to the web engine (geckoview).
Okay, thank you for considering this feature.