zh_CN.UTF-8 Character Set
xinmodao opened this issue · comments
NVIM v0.10.0
Build type: Release
LuaJIT 2.1.1713773202
system vimrc file: "$VIM/sysinit.vim"
fall-back for $VIM: "/usr/share/nvim"
lazy:·require("lazy.health").check()
lazy.nvim
-·OK·Git·installed
-·OK·no·existing·packages·found·by·other·package·managers
-·OK·packer_compiled.lua·not·found
-·WARNING·{alpha-nvim}:·overriding·
-·WARNING·{headlines.nvim}:·overriding·
-·ERROR·Issues·were·reported·when·loading·your·specs:
-·WARNING·{precognition.nvim}:·setting·a·table·to·Plugin.config·is·deprecated.·Please·use·Plugin.opts·instead
Error·detected·while·processing·CursorMoved·Autocommands·for·"*":
Error·executing·lua·callback:·...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:137:·attempt·to·index·a·nil·value
stack·traceback:
...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:137:·in·function·'build_virt_line'
...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:233:·in·function·'display_marks'
...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:261:·in·function·<...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:249>
-·WARNING·{precognition.nvim}:·setting·a·table·to·Plugin.config·is·deprecated.·Please·use·Plugin.opts·instead
this is a lazy message, change config to opts
we might need to update that in the documentation
Although, with that said. I cant reproduce the error exactly, what does your precognition config look like? and when did you last install/update it?
The character set I am using is zh_CN.UTF-8. When the line contains full-width characters or Chinese characters, I receive an error message. Does this mean it only supports half-width character sets?
Ill be honest, I a have only tested with UTF8 and the English alphabet, happy to take a look to support other characters/alphabets. can you give some example lines and any specific vim settings you have?
I might be encountering a similar issue
I'm using a CH-FR keyboard layout and since I'm using "à" instead of the carret "^" I'm getting this error as well
Config:
return {
"tris203/precognition.nvim",
config = {
hints = { Caret = { text = "à", prio = 2 } },
},
Error:
Error 10:19:36 msg_show.lua_error Error detected while processing CursorMoved Autocommands for "*":
10:19:36 msg_show Error executing lua callback: ...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:137: attempt to index a nil value
stack traceback:
...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:137: in function 'build_virt_line'
...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:233: in function 'display_marks'
...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:261: in function <...re/nvim/lazy/precognition.nvim/lua/precognition/init.lua:249>
This error is triggered once the vim cursor is on this char " - " (i.e. the start of a lua commented line)
If the vim cursor is on a blank space char the error is not triggered
Hope this help
Thank you for this amazing plugin :^)
EDIT:
I should have read more carefully, changing "config" to "opts" did the trick
EDIT2:
I've been a bit too fast to rejoice, it does trigger the error on Rust files, but not on Lua files
Thank you @monheimx9
This is brilliant recreation steps. I think I can see where the problem might be
I am not at my pc for a few hours but I will take a look later
I would imagine the lua/rust different is probably the indenting at the begining of the line. It's probably not trying to render the Caret character in the rust file
Can you test with the branch in the linked PR please @xinmodao @monheimx9
à is a multibyte character and is actually represented by 0xC3 0xA0. So you're seeing <c3>
and <a0>
as representations of these when we accidentally split the byte by inserting a new hint adjacent to the multibyte hint.
I remember @willothy and I discussing multibyte chars when we initially wrote the first PoC of precognition.
I need to look at how we build the virtual string and index into it. Maybe I can use vim functions instead of lua function to avoid splitting the byte.
I will do some more testing this evening
@monheimx9 Please could you update to the latest commit and try again
Yeah, maybe switching to vim.fn.strchar*
would help with this?
Yeah, maybe switching to
vim.fn.strchar*
would help with this?
i tried that, i have moved it round to build a table with the indices as the character positions, then concat the string at the end. It seems to work well
Just got home
It works pretty well, I can't reproduce any of the aforementioned behaviours/errors
Cheers :^)
Sorry, I just logged in today. I tested it and found that this problem occurs when a full-width character is followed by a half-width character. Full-width characters take up two characters, while half-width characters take up only one character, causing the problem
I can't see the image @xinmodao
Can you share your Precognition config please?
It looks like it's possible to reproduce the same issue using the precognition README.md file. Move the cursor to any characters in .nvim
in the first line of # 💭👀precognition.nvim
I tried to write a failing test in virtline_spec.lua
, but it doesn't seem to throw the exception for me. Here's what I had:
it("doesn't error on README.md header", function()
local cur_line = "# 💭👀precognition.nvim"
local cursorcol = 19
local line_len = vim.fn.strcharlen(cur_line)
local virt_line = precognition.build_virt_line({
w = hm.next_word_boundary(cur_line, cursorcol, line_len, false),
e = hm.end_of_word(cur_line, cursorcol, line_len, false),
b = hm.prev_word_boundary(cur_line, cursorcol, line_len, false),
Caret = hm.line_start_non_whitespace(cur_line, cursorcol, line_len),
Dollar = hm.line_end(cur_line, cursorcol, line_len),
}, line_len)
end)
I think this shows it well:
![Screen Shot 2024-05-31 at 21 15 53](https://private-user-images.githubusercontent.com/414204/335518024-de6b3a38-4210-4422-a6aa-f1daff4b8e1e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUxODU0NjUsIm5iZiI6MTcyNTE4NTE2NSwicGF0aCI6Ii80MTQyMDQvMzM1NTE4MDI0LWRlNmIzYTM4LTQyMTAtNDQyMi1hNmFhLWYxZGFmZjRiOGUxZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTAxJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkwMVQxMDA2MDVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02NmEyYjExNGViODJmOGI5OGVmODJkMjlmZTU2YWMyN2MwMzIwODRlMWY0ODYxYWNjMWViZjVlMWU2MzgyMmExJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.eYzCX3PQ2sYEteirnQd9Ee6uKbHgeUf8-e-iRBIAhsE)
ew
and E
and 2 columns short of where they should be, so there is an off-by-2 here, presumably due to the (incorrect?) treatment of the double width emoji chars. And b
is two in front of where it should be, for the same reason but in reverse.
Lualine seems to count the emoji as double columns (which I suspect is a bug). If I disable that, I get strange Nvim column reportings. On the first emoji I get 1,3
, but the second emoji is 1,7-5
, and all other characters after report a strange column value:
1,11-7
1,12-8
1,13-9
1,14-10
and so on in increasing pairs. Perhaps I don't understand what Nvim is trying to report here, or perhaps this is a Nvim bug. VS Code reports correct columns of 3 and 4 for each emoji (ie, a single column each).
Could the continued issues here be a Nvim char length bug?
Or, perhaps I'm well off base with all of the above :)