eframe: Native option mouse_passthrough makes window invisible

Question

eframe: Native option mouse_passthrough makes window invisible

Skrity opened this issue 2 years ago · comments

Describe the bug
When using mouse_passthrough native option the window is entirely transparent, it only shows in task bar and task view as a black box.
Also reproducible on master.

To Reproduce
Steps to reproduce the behavior:

use mouse_passthrough in some example app. Reproducible example follows.

main.rs

#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]

use eframe::egui;

fn main() {
    let options = eframe::NativeOptions {
        decorated: false,
        transparent: true,
        mouse_passthrough: true, // Changing this to true makes window fully invisible
        min_window_size: Some(egui::vec2(320.0, 100.0)),
        initial_window_size: Some(egui::vec2(320.0, 240.0)),
        ..Default::default()
    };
    eframe::run_native(
        "Basic Window",
        options,
        Box::new(|_cc| Box::new(MyApp::default())),
    );
}

#[derive(Default)]
struct MyApp {}

impl eframe::App for MyApp {
    fn clear_color(&self, _visuals: &egui::Visuals) -> egui::Rgba {
        egui::Rgba::TRANSPARENT
    }

    fn update(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
        egui::Window::new("My Window").show(ctx, |ui| {
            ui.label("Hello world");
        });
    }
}

cargo.toml

[package]
name = "hello_world"
version = "0.1.0"
edition = "2021"

[dependencies]
eframe = { version = "0.20.1" }

Expected behavior
Window will be visible, but cannot be interacted with.

Screenshots
Nothing to screenshot normally, but here's
screenshot of task view

Desktop (please complete the following information):

OS: Windows 10 21H2
Native via eframe
rustc 1.65.0
eframe - 0.20.1

Additional context
The original issue is here
There were also a report of this problem

Jay Oster · Answer 1 · Thu Jan 05 2023 07:24:37 GMT+0800 (China Standard Time)

This ... appears to be a winit bug. Using set_cursor_hittest() on Windows sets the WS_EX_LAYERED extended window attribute. Which is documented to have the following caveat:

After the CreateWindowEx call, the layered window will not become visible until the SetLayeredWindowAttributes or UpdateLayeredWindow function has been called for this window.

A call to the SetLayeredWindowAttributes function was removed in rust-windowing/winit#1815, even though this caveat was mentioned in rust-windowing/winit#1815 (comment). Importantly, this removal predates the introduction of set_cursor_hittest(), where the documented caveat was not revisited and so not accounted for in rust-windowing/winit#2232.

IMHO, this should be reported upstream to winit. This analysis is far from comprehensive but may provide some useful context.

edit: I used to have a section claiming it "works for me" on Windows 11. But this is not true. I missed the point about making a change to the minimal repro to actually reproduce the problem. Yes, it's a reproducible problem. But the repro code doesn't reproduce the problem as-is.

Skrity · Answer 2 · Thu Jan 05 2023 08:03:39 GMT+0800 (China Standard Time)

I'll report it to winit then, thank you. Also changed the code to avoid the confusion.

Jay Oster · Answer 3 · Thu Jan 05 2023 10:52:17 GMT+0800 (China Standard Time)

Thanks for fixing the minimal repro!

Could you also link back to this issue when you create the upstream one?

Skrity · Answer 4 · Thu Jan 05 2023 22:53:03 GMT+0800 (China Standard Time)

~~I have attempted to reproduce it in winit but couldn't: set_cursor_hittest() works fine on winit example window.~~
While trying to repeat the error I noticed that with_visible is somehow involved (it references #2279), because if it's set to true, set_cursor_hittest works as intended.

Current observation is that using set_cursor_hittest(false) while window in set_visible(false) breaks it.

Angelos · Answer 5 · Sat Apr 01 2023 22:22:41 GMT+0800 (China Standard Time)

Is there a workaround to this currently? I tried to set visible to true on the frame in my app but it still doesn't render anything. It would seem that I need to access the native window creation itself to do fix this manually (not sure). One way I imagine is to manually get a handle to the window (in windows) and pass the correct options.

Edit:

I developed a work around specific for windows. I use the windows api functions EnumWindows and SetLayeredWindowAttributes to make the window appear again. First I get the window handle through the EnumWindows, searching for the correct window name and then call SetLayeredWindowAttributes(handle, 0, 255, LWA_ALPHA) to make the window visible.

Here is my EnumWindows callback code for someone who also wants to get around this (using windows crate but winapi is identical or even more trivial):

extern "system" fn enum_win(hwnd: HWND, mut lparam: LPARAM) -> BOOL {
    let mut class_name = [0u16; 256];
    unsafe { GetWindowTextW(hwnd, &mut class_name) };
    let class_name = String::from_utf16_lossy(&class_name[..6 as usize]);
    println!("class_name: {}", class_name);
    let parameter = unsafe {std::mem::transmute::<LPARAM, &mut LPARAM>(lparam)};
    if class_name == "name" {
        println!("Found window: {}, {}", hwnd.0, class_name);
        (*parameter).0 = hwnd.0;
        return BOOL(0)
    }
    return BOOL(1)
}

Nonetheless, the bug remains to be fixed as it completely removes the mouse_passthrough functionality.

Skrity · Answer 6 · Mon Apr 03 2023 03:22:05 GMT+0800 (China Standard Time)

For myself I worked around it in eframe.
In file native/run.rs you've got to move this code:

if self.native_options.mouse_passthrough {
    gl_window.window().set_cursor_hittest(false).unwrap();
}

from init_run_state() to GlowWinitApp method paint() right after integration.post_present(window); as this function is settings visibility back to window. E.g.

integration.post_present(window);
window.set_cursor_hittest(!self.native_options.mouse_passthrough).unwrap();

mouse_passthrough works properly after this change.

ARYA · Answer 7 · Sat May 20 2023 02:27:04 GMT+0800 (China Standard Time)

For myself I worked around it in eframe. In file native/run.rs you've got to move this code:
if self.native_options.mouse_passthrough {
    gl_window.window().set_cursor_hittest(false).unwrap();
}
from init_run_state() to GlowWinitApp method paint() right after integration.post_present(window); as this function is settings visibility back to window. E.g.
integration.post_present(window);
window.set_cursor_hittest(!self.native_options.mouse_passthrough).unwrap();
mouse_passthrough works properly after this change.

Is there any update on this issue or rather a cleaner way of approaching patching the package?

Markus Gauwatz · Answer 8 · Tue Oct 24 2023 07:31:24 GMT+0800 (China Standard Time)

if the mentioned "workaround" of @Skrity is the fix, why this isn't merged yet?

Cory Robertson · Answer 9 · Mon Nov 20 2023 13:42:53 GMT+0800 (China Standard Time)

For myself I worked around it in eframe. In file native/run.rs you've got to move this code:
if self.native_options.mouse_passthrough {
    gl_window.window().set_cursor_hittest(false).unwrap();
}
from init_run_state() to GlowWinitApp method paint() right after integration.post_present(window); as this function is settings visibility back to window. E.g.
integration.post_present(window);
window.set_cursor_hittest(!self.native_options.mouse_passthrough).unwrap();
mouse_passthrough works properly after this change.

I cant quite seem to re-create your workaround, do you have a fork of this change?

Emil Ernerfeldt · Answer 10 · Wed Nov 22 2023 01:30:44 GMT+0800 (China Standard Time)

Is this still a problem on latest master? The code has been significantly rewritten over the last week

Cory Robertson · Answer 11 · Wed Nov 22 2023 02:09:14 GMT+0800 (China Standard Time)

Is this still a problem on latest master? The code has been significantly rewritten over the last week

For some reason, I didn't think to check against master, and instead tried checking only the crates version. That's on me! If there is any work that needs to be done relating to this, I'd love to help, such as an example overlay program maybe? I'm mostly new to this though. 😄

Edit: After using what is current on master branch, the only thing I found that was a little unpleasant was that egui/crates/eframe/src/epi.rs clear_color(&egui::Visuals) only using the hard coded value, is there a reason? Because the function even has a comment.
// _visuals.window_fill() would also be a natural choice
After patching that to instead use the window_fill field, it seems to work perfectly though to my enjoyment!

Kekschen · Answer 12 · Sat Jun 29 2024 21:28:12 GMT+0800 (China Standard Time)

Hey, is there any progress on this? I know it's only been an issue for around one and a half years and it is a breaking bug, but shouldn't this get some patch here maybe if upstream isn't doing anything about it?

I suggest a temporary feature flag that fixes this locally and can be removed later.

Kekschen · Answer 13 · Sun Jun 30 2024 18:53:49 GMT+0800 (China Standard Time)

Is there a workaround to this currently? I tried to set visible to true on the frame in my app but it still doesn't render anything. It would seem that I need to access the native window creation itself to do fix this manually (not sure). One way I imagine is to manually get a handle to the window (in windows) and pass the correct options.

Edit:

I developed a work around specific for windows. I use the windows api functions EnumWindows and SetLayeredWindowAttributes to make the window appear again. First I get the window handle through the EnumWindows, searching for the correct window name and then call SetLayeredWindowAttributes(handle, 0, 255, LWA_ALPHA) to make the window visible.

Here is my EnumWindows callback code for someone who also wants to get around this (using windows crate but winapi is identical or even more trivial):
extern "system" fn enum_win(hwnd: HWND, mut lparam: LPARAM) -> BOOL {
    let mut class_name = [0u16; 256];
    unsafe { GetWindowTextW(hwnd, &mut class_name) };
    let class_name = String::from_utf16_lossy(&class_name[..6 as usize]);
    println!("class_name: {}", class_name);
    let parameter = unsafe {std::mem::transmute::<LPARAM, &mut LPARAM>(lparam)};
    if class_name == "name" {
        println!("Found window: {}, {}", hwnd.0, class_name);
        (*parameter).0 = hwnd.0;
        return BOOL(0)
    }
    return BOOL(1)
}
Nonetheless, the bug remains to be fixed as it completely removes the mouse_passthrough functionality.

And besides your code being not a general workaround, it's also just bad.

I'd just recommend implementing something similar to this post: https://stackoverflow.com/a/2620522

static INIT_WINDOW_VISIBILITY_FIX: Once = Once::new();

pub fn init_window_visibility_fix() {
    INIT_WINDOW_VISIBILITY_FIX.call_once(|| {
        if let Err(e) = fix_window_visibility() {
            log::error!("Failed to fix window visibility: {:?}", e);
        }
    });
}

fn fix_window_visibility() -> windows::core::Result<()> {
    unsafe { EnumWindows(Some(enum_window_proc), LPARAM(0)) }
}

extern "system" fn enum_window_proc(hwnd: HWND, _: LPARAM) -> BOOL {
    let mut class_name_buffer = [0u16; MAX_PATH as usize];

    unsafe {
        let mut process_id: u32 = 0;
        GetWindowThreadProcessId(hwnd, Some(&mut process_id));
        if process_id == GetCurrentProcessId() {
            if GetWindowTextW(hwnd, &mut class_name_buffer) == 0 {
                return TRUE;
            }

            let _ = SetLayeredWindowAttributes(hwnd, COLORREF(0), 255, LWA_ALPHA);
        }
    }
    TRUE
}

I just hope someone pushes some fixes to winit at some point

Emil Ernerfeldt · Answer 14 · Sun Jun 30 2024 20:34:27 GMT+0800 (China Standard Time)

I just hope someone pushes some fixes to winit at some point

You could be that someone!