nvim-neorg / tree-sitter-norg

A TreeSitter parser for the Neorg File Format

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Escape characters shouldn't be highlighted in inline math.

linguini1 opened this issue · comments

The norg spec says that math can be written inline with a leading and trailing $, or it can be written across
multiple lines using the @math, @end tags.

Highlights work correctly in the @math tags, but not in inline math. I believe since the norg spec says that math tags should be LaTeX typeset, inline math should also support LaTeX typeset. But, LaTeX is full of \ characters, which are being highlighted as escape characters inside inline math. It would be better if they were ignored inside inline math.

Inline math with highlighted escape characters

inline-math

Math tag math highlighted properly

tag-math

Hey!

This is actually intentional. It seems very counterintuitive at first, but I'll try to lay out our thought process here.

The TL;DR is: for any slightly more sophisticated math use $|Hello, \LaTeX!|$ instead.

I think I discussed this with mrossinek at least five times by now lol, but allowing escape characters in verbatim markup allows for quite a few things. To give a full answer, a distinction between the regular and free-form attached modifiers must be made. Each attached modifier (like bold, italic, maths, verbatim etc.) by default are in their regular state - they must immediately enclose some text, i.e. * this * or $ this $ are not allowed. The escape character is allowed to exist within any of these when they're in their regular state: $this \$ is allowed$. To make the attached modifier free-form, you must not only enclose the text in your modifier of choice but also in pipes, for instance: *|this|*. When this happens, whitespace is allowed around the markup, and in verbatim attached modifiers escape characters are not parsed.
Examples include: *| look ma, whitespace! |*, $|Look at all my \LaTeX!|$.

The way I like to think of it are lua regular strings and multiline strings. Regular strings can also contain the string character if it's escaped, and the multiline string syntax consists of two characters and the escape character is treated as verbatim within those. Analogously to Norg, this means you can do things like: `this is\` a long singular verbatim!`, but the following will display the backslash: `|this is\` a long singular verbatim|`.

There are more technical reasons, like more uniform precedence, but I'll leave those to my work in progress "design decisions" document where I talk about every decision that was made in Norg.
For now, just know that when dealing with inline maths you should use just $ for very simple situations ($f(x) = x^2$ or $Hello, \\LaTeX$ - notice the double backslash) and $|...|$ for all others :D

@vhyrro This makes so much sense, and I had no idea I could add the pipes in this scenario. Do you know where in the Neorg help docs I can find this? I can't find any mention of the pipes in the help docs. If it's not there, I'd love to open a PR to add it with the math help because I think others might find this useful.

I believe the pipes are only mentioned in the :h neorg-cheatsheet, but are not mentioned in the :h neorg document. I think it's cause the pipe syntax was added long after that document was made haha. A PR would be greatly appreciated!

I believe the pipes are only mentioned in the :h neorg-cheatsheet, but are not mentioned in the :h neorg document. I think it's cause the pipe syntax was added long after that document was made haha. A PR would be greatly appreciated!

Interestingly, Neovim tells me it can't find that help tag! I do see a small preview of it when I use the telescope help-tag finder, but I can't open it. I can definitely open a PR to add it to the :h neorg document though, and then maybe another if I can figure out this :h neorg-cheatsheet issue.

telescope capture