carbon-steel / detour.nvim

Repro:

use https://github.com/folke/zen-mode.nvim
enter zen-mode by issuing :ZenMode.
issue :Detour

Error message:

E5108: Error executing lua: ...s/.local/share/nvim/lazy/detour.nvim/lua/detour/init.lua:154: attempt to perform arithmetic on a nil value
stack traceback:
	...s/.local/share/nvim/lazy/detour.nvim/lua/detour/init.lua:154: in function 'nested_popup'
	...s/.local/share/nvim/lazy/detour.nvim/lua/detour/init.lua:188: in function 'popup'
	...s/.local/share/nvim/lazy/detour.nvim/lua/detour/init.lua:255: in function 'Detour'
	/Users/neilvyas/.config/nvim/init.lua:53: in function </Users/neilvyas/.config/nvim/init.lua:52>

Other popups, like :Lazy, still work from within zen-mode. Detour is very useful (or, will be, once this bug is addressed) in conjunction with zen-mode since zen-mode exits automatically when attempting to navigate away from its window or on many other triggers; popups do not exit it. So, I wanted to use Detour to have access to neogit from within zen-mode.

I haven't done any debugging yet, other than a cursory examination of the reported error site. I'm just logging this issue now so that I don't forget to!

now, this is more likely to be caused by neogit, but I've uncovered another issue:

vim.keymap.set("n", "<Leader>g", function ()
    require("detour").Detour()

    vim.cmd.enew()
    vim.bo.bufhidden = 'delete'

    require("neogit").open({ kind = "replace"})
end)

this keybind only works the first time; every time after that, you get a blank buffer (if the middle two lines are omitted, then a copy of the current buffer), and you cannot get neogit to open again even outside of a Detour (say, by using :Neogit, or :lua require("neogit").open({kind = "tab"}))

so, the above is a detour--related issue---it doesn't happen when you just use neogit on its own. That is, require("neogit").open({ kind = "replace"}) works arbitrarily many times.

Ok, so I've done a bit of debugging.

Take this line, identified as the source of the error:

https://github.com/carbon-steel/detour.nvim/blob/e8a9b09610b30269afe4aab07787d8167e807ff7/lua/detour/init.lua#L154C1-L154C75

As far as I can tell, this will never work, because the popup_to_layer table is only updated when the detour is destroyed, never when it's constructed. There seem to be a few ways to fix this: (a) just remove popup_to_layer entirely, and instead rely on vim.api.nvim_win_get_config(parent).zindex + 1 to set the zindex directly; (b) insert into popup_to_layer appropriately at the callsite https://github.com/carbon-steel/detour.nvim/blob/e8a9b09610b30269afe4aab07787d8167e807ff7/lua/detour/init.lua#L187C1-L190C8 ; or (c) just take this data as an argument and thread it around.

I can't understand what purpose popup_to_layer even serves, so I think that approach (a) makes the most sense. However, I don't know any lua, really, and I don't know very much about the nvim api or model; I just tried to understand the issue directly in front of my eyes, rather than worrying too much about discovering its true root cause or anything like that.

Let me know if this seems reasonable and I can take a swing at implementing it and updating the test coverage.

Hi, it looks like you've done a lot of investigation into this issue (thank you!). It'll take me some time to fully respond to what you've said (and I may need a day or two to do so), but here are a few preliminary things I can mention right off the bat:

because the popup_to_layer table is only updated when the detour is destroyed, never when it's constructed

I am not sure what you mean. You can see that popup_to_layer is updated:

here
and here

just remove popup_to_layer entirely, and instead rely on vim.api.nvim_win_get_config(parent).zindex + 1

I like this idea. If this can allow us to remove popup_to_layer entirely, that would be better. Less state is always better.

I don't think it will make a difference, but just to be sure, could you try using the plugin on the dev branch? It's the most up-to-date code and has some bug fixes.

Seems that nested popups just never work?

That's very surprising to hear that nested popups "never work". Could you try this:

git clone this plugin to your local computer.
Open neovim with the command nvim --clean --cmd "set rtp+=/path/to/detour.nvim" (of course, you'll have to replace /path/to/detour.nvim with your own path). This opens neovim without configs, plugins, or shada except for the detour.nvim plugin. Reproducing the issue from a clean neovim will help us confirm that the issue is indeed with the plugin and not other configs/plugins/state.
[Optional] The default colorscheme makes popups look bad. Change it to :colorscheme industry
Do :Detour
Do another :Detour
It should look like this (Notice the two borders, one for each popup):

This should work whether you're on the main branch (which is the default) or the dev branch. If you find nested popups are not working for you, could you try writing down steps to reproduce the error, using a clean neovim as above?

Could you share the first three lines of output when you call nvim --version? I'd like to know what version of neovim you're using.

I don't want to discourage you from trying to solve the bug yourself, but I think it'd be worth it to discuss things a bit more before either of us has a go at coding the solution. I'll investigate the initial Zen issue tomorrow or the day after.

I am not sure what you mean. You can see that popup_to_layer is updated:

I think what I meant was that both of those updates happen after when we make the access which causes the error. That is, in the scenario I outlined, the first interaction with (a specific key in) popup_to_layer is a read, rather than an initialization.

That's very surprising to hear that nested popups "never work". Could you try this:

Sooooooooooo I was definitely overzealous in saying so. Repeated use of :Detour definitely works for me. I apologize for making you check this! I've been quite sleep deprived recently so I have no idea what motivated me to generalize without evidence like this.

Could you share the first three lines of output when you call nvim --version? I'd like to know what version of neovim you're using.

NVIM v0.9.2
Build type: Release
LuaJIT 2.1.0-beta3

I'll try out the plugin on the dev branch sometime today and get back to you. Thanks for your reply, and for the plugin!

wait... it was staring me in the face.

I think what I meant was that both of those updates happen after when we make the access which causes the error. That is, in the scenario I outlined, the first interaction with (a specific key in) popup_to_layer is a read, rather than an initialization.

To avoid the error, you need to code you referenced to have run. But that has only run for the parent when it is a detour popup, meaning that successive use of :Detour will work, but any other popup will break, because it will trigger the nested_popup codepath with a parent window which hasn't run any of the code you referenced.

Sorry for explaining it poorly; again, I cite sleep deprivation.

(Successive Detours are fine because of the first site you referenced,

detour.nvim/lua/detour/init.lua

Line 156 in e8a9b09

popup_to_layer[child] = popup_to_layer[parent] + 1

)

the dev branch doesn't address the problem, I've just tested locally. I figure I'll try updating the tests to include this case and then attempt resolution myself using option (a), if only just to get som epractice developing plugins and with lua.

Thanks for offering to fix the issue yourself. I'll be happy to work with you on this. Please make sure that your changes are on top of the dev branch :)

Detour from `:ZenMode` or `:Lazy` results in error