Better syntax highlighting for scribble files

[Ramón L. Panadés-Barrueta]

Hello and thanks a lot for this great project.

I have been doing some literate programming in Racket using scribble. Everything works perfectly except for the syntax highlighting of the scribble files. Here is an example of what I mean:

Is it possible to remove the highlighting of keywords on the plain text regions of the scribble files? I am aware of the scribble-mode, but if I use it, I lose the editing power provided by racket-mode. I have also taken a look at some related issues like #64 and #206, but they did not help me much.

For reference:

(λ . loren) racket --version && emacs --version
Welcome to Racket v8.9 [cs].
GNU Emacs 30.0.50

Thanks in advance.

[Greg Hendershott]

Thanks for the question and the kind words.

The current plan is to support this better via a new racket-hash-lang-mode --- but that's currently on the hash-lang branch and not merged.

There are still some things to test in the underlying mode, as well as figure out how to fit in with both the Racket and Emacs ecosystems, as happens to be discussed pretty recently in #661.

[Ramón L. Panadés-Barrueta]

@greghendershott thanks a lot! I will take a look at that branch.

[Greg Hendershott]

FWIW I've dog-fooding that branch heavily lately. Used to edit a .scrbl file as well as many .rkt files.

I'm trying to pin down enough details to merge it to main branch, as soon as I can.

[Ramón L. Panadés-Barrueta]

Hey @greghendershott I have been using the hash-lang branch for a while, and it works really well. Thanks for all the work. I have a question though. When I activate the hash-lang mode with M-x racket-hash-lang-mode the syntax highlighting of the Racket code inside Scribble's examples disappear:

Is this something to be expected? For reference, this is how it looks in the default racket-mode:

[Greg Hendershott]

@Panadestein Thanks for trying it and letting me know your experience.

I think this explanation is relevant so I'll copy it:

There are two levels of coloring happening.

A lang can supply a "color-lexer". On the one hand a lexer is fast, and only cares about errors like mismatched string quotes (not, say, mismatched parens). On the other hand, a lexer only knows about tokens -- strings, numbers, comments, etc.

So the basic coloring, similar to what you'd see in DrRacket, is just tokens. It is fairly plain.

In your example screenshot I do see numbers colored, so I think that's working?

Because I prefer more "garish" colors, I added the ability for racket-xp-mode to contribute more colors. Basically, each end of an arrow you'd see in DrRacket (binding or use) times each kind of arrow (lexical, imported, imported from module-language) = six faces. You can customize these faces, but the defaults emphasize binding (definition or require) sites, as well as uses of imports. This is roughly "as colorful" as classic racket-mode, but not exactly the same. This is a relatively recent experiment, last couple weeks, and may need some fine-tuning. I think it works OK-ish for racket and scribble files I've dog-fooded; @samth has pointed out to me it's not ideal for rhombus import aliases.

Some people would call this "semantic" highlighting, as opposed to the fast lexer "syntax" highlighting.

Anyway, the racket-xp-mode check-syntax takes longer -- so the colors it contributes show up later. Also an expansion error can mean it doesn't understand enough to contribute any colors.

Although I tried to explain some of this in the doc strings, and in the "guide" explanation I linked to above, maybe it's still not clear. Even if it's clear, it still may feel surprising and not satisfactory? Also maybe I'm misunderstanding and you're experiencing something else?

In your case:

0. Keep in mind classic racket-mode is nicer here only because your @examples happens to be racket code; otherwise that feature would be a bug. :)
1. racket-hash-lang-mode is coloring your numbers. But your colors for numbers are different. Maybe you want to customize font-lock-constant-face to be "orange-ish" instead of dark purple?
2. If you also enable the minor mode racket-xp-mode in the buffer, you should see more colors (not exactly the same as classic racket-mode but roughly "as colorful").

• Those extra colors will appear only after you pause typing long enough for a check-syntax to run and give results (for some files this is fairly fast, for others it could take seconds).

---

If some detail I said seems wrong, please let me know.

If racket-xp-mode doesn't help as much as you'd want, please let me know.

And do let me know any thoughts, including subjective opinions positive or negative.

[Greg Hendershott]

p.s. Oh. Actually racket-xp-mode won't help contribute more colors here, because check-syntax doesn't draw any arrows inside examples forms. 😞

So things like define in your screenshot are only lexed as "symbol", end of story. (racket-xp-mode doesn't see them as identifiers from the check-syntax analysis, and doesn't add more colors for them.)

(By default racket-hash-lang-mode uses the Emacs default face for things lexed as "symbol", because that can be noisy and not convey much -- nearly everything that isn't a string or number or delimiter is a symbol. But you can change that via the customization variable racket-hash-lang-token-face-alist.)

[Ramón L. Panadés-Barrueta]

Hey @greghendershott thanks a lot for the throughout explanation. At the end of the day, using racket-mode is more than enough. After all, the Scheme code I am writing is the important part :) I can switch to hash-lang when I want to write some text. The package work very well the way it is. Thanks again, and feel free to close this issue, at least from my side.