Issues found with syntax hightlighting

[Santos Gallegos]

I have been using the highlight feature for a couple of weeks now (python mainly).

There are three issues I have found so far:

• While typing some syntax may be invalid, and everything goes red (fixed with highlight link TSError Normal)

• Sometimes there is an error (not sure how to replicate), and the text gets replaced with an error from lua. This is fixed by reloading the text with :e. Fixed, haven't experienced this anymore.

• When the using folds, some parts of the code doesn't get highlighted. This happens when I open a file where the text is folded (syntax is highlighted as normal), then go to the last line with G (syntax isn't highlighted, the whole text is white). This is fixed by forcing a redraw with <C-L>.

[Kiyan]

this is an issue with neovim's api. A refactoring was coming up but got delayed because the author is not available. We will try to fix that issue ourselves, we'll keep this issue updated in the meantime

[Stephan Seitz]

Hi, the ERROR highlight is easily fixable:

    require "nvim-treesitter".setup()
    require "nvim-treesitter.highlight"
    local hlmap = vim.treesitter.TSHighlighter.hl_map

    --Misc
    hlmap.error = nil

We probably should default to that. Then, @error is not highlighted for you. hlmap defines how the tree-sitter queries are translated to Vim hightlight groups. I maintain my own mapping:

    require "nvim-treesitter.highlight"
    local hlmap = vim.treesitter.TSHighlighter.hl_map

    --Misc
    hlmap.error = nil
    hlmap["punctuation.delimiter"] = "Delimiter"
    hlmap["punctuation.bracket"] = "Delimiter"

    -- Constants
    hlmap["constant"] = "Constant"
    hlmap["constant.builtin"] = "Type"
    hlmap["constant.macro"] = "Define"
    hlmap["string"] = "String"
    hlmap["string.regex"] = "String"
    hlmap["string.escape"] = "SpecialChar"
    hlmap["character"] = "Character"
    hlmap["number"] = "Number"
    hlmap["boolean"] = "Boolean"
    hlmap["float"] = "Float"

    -- Functions
    hlmap["function"] = "Function"
    hlmap["function.builtin"] = "Special"
    hlmap["function.macro"] = "Macro"
    hlmap["parameter"] = "Identifier"
    hlmap["method"] = "Function"
    hlmap["field"] = "Identifier"
    hlmap["property"] = "Identifier"
    hlmap["constructor"] = "Type"

    -- Keywords
    hlmap["conditional"] = "Conditional"
    hlmap["repeat"] = "Repeat"
    hlmap["label"] = "Label"
    hlmap["operator"] = "Operator"
    hlmap["keyword"] = "Repeat"
    hlmap["exception"] = "Exception"
    hlmap["include"] = "Include"
    hlmap["type"] = "Type"
    hlmap["type.builtin"] = "Type"
    hlmap["structure"] = "Structure"

The issues that you're observing are problems introduced by the last runtime update of Neovim built-in treesitter. Apparently, the code that's performing the regex queries still has some bugs.

[Steven Sojka]

Would this be causing this issue I'm seeing?

[Thomas Vigouroux]

Yep, both of them are related, the way you can help is tell us the edit you did to cause that. Because that's actually the tree getting out of sync.

[Steven Sojka]

@vigoux Sounds good. I'm getting it a lot on typescript code. Seems to be on just inserting text. Is this something we can fix or does it need to be fixed upstream in nvim?

[Thomas Vigouroux]

It needs to be fixed upstream. And it is related to especially gnarly parts of neovim, thus it requires quite q bit of effort (at least for me...) to fix.

[Raphael]

Same issue here. I found it has some problems with syntax. I defined some syntax group in my colorscheme. when i install treesitter. It didn't follow syntax group,It even changed the color scheme.

before:

after:

With typescript. It works better than default regexengine2 ,But it looks like I didn't get more color schemes.

before:

after:

[Thomas Vigouroux]

@glepnir

Treesitter highlighting is a completely different mechanism than standard highlighting. tbh they don't even share any single line of code (on the implementation side). Thus if you define syntax groups, treesitter will just not know that, you have to use queries to do so.

The colors changed and that's normal, treesitter highlighting tries to bring meaning together with colors, here for example, in red you find keywords, green are function calls and so on. So the colorscheme changed yes, but if you compare you go code and you typescript code, they look uniform and consistent on the highlighting side. To customize the behaviour though, you can customize the highlight groups nvim-treesitter uses using :highlight link {TSHlGroup} {TheGroupYouWant}.

[Raphael]

@vigoux Aha thanks. About this highlight link {TSHlGroup} {TheGroupYouWant}. I think i know the {TheGroupYouWant} that the syntax group i defined in my colorscheme of ts or other language syntax group. This {TSHlGroup} mean these?

  require "nvim-treesitter.highlight"
    local hlmap = vim.treesitter.TSHighlighter.hl_map

    --Misc
    hlmap.error = nil
    hlmap["punctuation.delimiter"] = "Delimiter"
    hlmap["punctuation.bracket"] = "Delimiter"

    -- Constants
    hlmap["constant"] = "Constant"
    hlmap["constant.builtin"] = "Type"
    hlmap["constant.macro"] = "Define"
    hlmap["string"] = "String"
    hlmap["string.regex"] = "String"
    hlmap["string.escape"] = "SpecialChar"
    hlmap["character"] = "Character"
    hlmap["number"] = "Number"
    hlmap["boolean"] = "Boolean"
    hlmap["float"] = "Float"

    -- Functions
    hlmap["function"] = "Function"
    hlmap["function.builtin"] = "Special"
    hlmap["function.macro"] = "Macro"
    hlmap["parameter"] = "Identifier"
    hlmap["method"] = "Function"
    hlmap["field"] = "Identifier"
    hlmap["property"] = "Identifier"
    hlmap["constructor"] = "Type"

    -- Keywords
    hlmap["conditional"] = "Conditional"
    hlmap["repeat"] = "Repeat"
    hlmap["label"] = "Label"
    hlmap["operator"] = "Operator"
    hlmap["keyword"] = "Repeat"
    hlmap["exception"] = "Exception"
    hlmap["include"] = "Include"
    hlmap["type"] = "Type"
    hlmap["type.builtin"] = "Type"
    hlmap["structure"] = "Structure"

[Thomas Vigouroux]

Hmm you have a pretty old version of the plugin, I mean these. Update to latest master (on both neovim and the plugin) so that you can configure this how you like it.

[Raphael]

Ok. I got it. I will try to define them. Thanks .

[Raphael]

@PitcherTear22 It's my plugin spaceline.vim

[Santos Gallegos]

Just an update here, the only problem that isn't fixed is

When the using folds, some parts of the code doesn't get highlighted. This happens when I open a file where the text is folded (syntax is highlighted as normal), then go to the last line with G (syntax isn't highlighted, the whole text is white). This is fixed by forcing a redraw with .

I have updated the description

[Santos Gallegos]

I can't replicate this anymore with folding, but it still happens with other cases like #235. I'm closing this in favor of the other issue.

[Jacob Duncan]

Same issue here. I found it has some problems with syntax. I defined some syntax group in my colorscheme. when i install treesitter. It didn't follow syntax group,It even changed the color scheme.

before:

after:

With typescript. It works better than default regexengine2 ,But it looks like I didn't get more color schemes.

before:

after:

What colorscheme and font is this?