Igalia / wolvic

Is your feature request related to a problem? Please describe.
TV apps (i.e. web apps designed for large screen devices) look great in a VR environment (like the Wolvic browser). However, most can only be controlled with the navigation keys (i.e. LEFT/RIGHT/UP/DOWN/ENTER/BACK). Currently, these apps cannot be used within the Wolvic browser, because only mouse support seems to be integrated.

Describe the solution you'd like
Is it possible to provide an option to enable spatial navigation by mapping the AR/VR controller D-pad input to LEFT/RIGHT/UP/DOWN key events?
I have already noticed that ENTER and BACK are already mapped to key events on an Oculus Quest 2 controller.

Describe alternatives you've considered
n/a

Additional context
Here is an example TV web app that can be used with mouse and keyboard: http://msx.benzac.de?scale=default&zoom=auto
Although this app can be controlled entirely with mouse, spatial navigation with a controller would be desirable.
For more information about this app, please see this info page: https://msx.benzac.de/info/

We do handle MotionEvents generated from the D-pads and pass it to the web engine (gecko), and I can see it works at least when you want to scroll the web page with D-pads.

wolvic/app/src/common/shared/com/igalia/wolvic/browser/api/WPanZoomController.java

Lines 18 to 25 in f9b69c1

    
               /** 
        
                * Process a non-touch motion event through the pan-zoom controller. Currently, hover and scroll 
        
                * events are supported. Pointer coordinates should be relative to the display surface. 
        
                * 
        
                * @param event MotionEvent to process. 
        
                */ 
        
               void onMotionEvent(final @NonNull MotionEvent event); 
        
           }

And unfortunately, this is something related to the web engine API, I'm not sure if we can do more for it, but let's see.

Thank you for your quick response.

I think a solution approach would be to select in an options panel whether a handleScrollEvent should dispatch an actual scroll event or a keyEvent.

For example, instead of a scroll-up, the following code could be executed (similar to the handleBack function dispatching a KeyEvent.KEYCODE_BACK event).

dispatchKeyEvent(new KeyEvent(KeyEvent.ACTION_DOWN, KeyEvent.KEYCODE_DPAD_UP));
dispatchKeyEvent(new KeyEvent(KeyEvent.ACTION_UP, KeyEvent.KEYCODE_DPAD_UP));

wolvic/app/src/common/shared/com/igalia/wolvic/VRBrowserActivity.java

Line 1124 in f9b69c1

    
           void handleScrollEvent(final int aHandle, final int aDevice, final float aX, final float aY) {

I tried to do as you proposed, #1113 But unfortunately that only works for text-editing. I guess we may still need some kind of API to directly pass the keyboard events to the web engine (geckoview).

Okay, thank you for considering this feature.

	/**
	* Process a non-touch motion event through the pan-zoom controller. Currently, hover and scroll
	* events are supported. Pointer coordinates should be relative to the display surface.
	*
	* @param event MotionEvent to process.
	*/
	void onMotionEvent(final @NonNull MotionEvent event);
	}

Spatial navigation support for AR/VR controllers with D-pads