code is not displayed correctly

Question

code is not displayed correctly

Dr-kyle opened this issue 2 years ago · comments

my code is:

// 18.2.0
import { useEffect } from "react"
// 11.8.0
import hljs  from 'highlight.js'
// 5.0.5
import { marked } from "marked";
// 1.0.2
import { markedHighlight } from "marked-highlight";
// 1.0.1
import { mangle } from "marked-mangle";

export default function Index() {

  useEffect(() => {
    marked.use({ headerIds: false,}, mangle(), markedHighlight({
      langPrefix: 'hljs language-',
      highlight(code, lang) {
        const language = hljs.getLanguage(lang) ? lang : 'plaintext';
        return hljs.highlight(code, { language }).value;
      }
    }));
    let res = marked.parse('# Marked in the browser\n\nRendered by **marked**.\n```java\nInteger a = 3;\n```\n');
    console.log(res)
    document.getElementById('test-m').innerHTML = res;
  }, [])
  return (
    <>
      <div id="test-m"></div>
    </>
  )
}

result:

Can you help me point out where I am wrong?

Tony Brix · Answer 1 · Sun May 14 2023 03:13:15 GMT+0800 (China Standard Time)

Seems like it is something with react. Removing react seems to make it work.

https://runkit.com/embed/uq1ebjn57tfp

icebaker · Answer 2 · Mon May 29 2023 08:06:52 GMT+0800 (China Standard Time)

The same problem exists with Svelte. The deprecated way still works perfectly fine:

marked.setOptions({
  highlight: function(code, language) {
    const validLanguage = hljs.getLanguage(language) ? language : 'plaintext';
    return hljs.highlight(code, { language: validLanguage }).value;
  }
});

Content:

```elixir
IO.puts("Hello, world!")
```

Generates:

<pre><code class="language-elixir"><span class="hljs-title class_">IO</span>.puts(<span class="hljs-string">&quot;Hello, world!&quot;</span>)
</code></pre>

The new way:

marked.use(markedHighlight({
  langPrefix: 'hljs language-',
  highlight(code, lang) {
    const language = hljs.getLanguage(lang) ? lang : 'plaintext';
    return hljs.highlight(code, { language }).value;
  }
}));

Content:

```elixir
IO.puts("Hello, world!")
```

Generates:

<pre><code class="hljs language-elixir">&lt;span class=<span class="hljs-string">&quot;hljs-title class_&quot;</span>&gt;<span class="hljs-title class_">IO</span>&lt;/span&gt;.puts &lt;span class=<span class="hljs-string">&quot;hljs-string&quot;</span>&gt;&amp;quot;<span class="hljs-title class_">Hello</span>, world!&amp;quot;&lt;/span&gt;
</code></pre>

Tony Brix · Answer 3 · Mon May 29 2023 08:45:39 GMT+0800 (China Standard Time)

Can you create a repo that reproduces this?

icebaker · Answer 4 · Tue May 30 2023 07:41:19 GMT+0800 (China Standard Time)

So, it's a weird thing. While trying to reproduce it in a new repo, I realized where it's happening, and it appears to be unrelated to the new way or the legacy way.

The code is here: https://github.com/icebaker/myapp

Here's what's going on: If you update your code while your server is in development mode, the page automatically updates to reflect your changes. But for some reason, this seems to mess up the code. A simple page refresh brings things back to normal. I've captured this in a video:

behavior.mp4

Note that, for some reason, this only happens with marked (legacy or new way). If I'm using highlight.js purely, it does not mess up the rendering.

Tony Brix · Answer 5 · Tue May 30 2023 11:46:23 GMT+0800 (China Standard Time)

This looks like an issue in svelte. marked returns the correct html so there is nothing we can change in marked or marked-highlight to fix this.

icebaker · Answer 6 · Wed May 31 2023 06:52:56 GMT+0800 (China Standard Time)

Alright, digging deeper into debugging, I've discerned the root cause. Indeed, it's a problem linked to the new technique, yet it doesn't pertain to marked-highlight. Instead, the issue stems from the shift from setOptions to use.

With setOptions, we have the capability to invoke it multiple times, and it will invariably replace the existing option.

On the other hand, with use, if it's invoked a second time, it will append the extension twice. If invoked yet again, it's added a third time, and so forth.

That's the reason it "incorrectly" renders the code—it applies a highlight onto an already highlighted portion.

When dealing with setOptions:

marked.setOptions({highlight: ...});
marked.parse(code);
// IO.puts("Hello, world!")

marked.setOptions({highlight: ...});
marked.parse(code);
// IO.puts("Hello, world!")

marked.setOptions({highlight: ...});
marked.parse(code);
// IO.puts("Hello, world!")

When dealing with marked.use:

marked.use(markedHighlight({...}));
marked.parse(code);
// IO.puts("Hello, world!")

marked.use(markedHighlight({...}));
marked.parse(code);
// <span class="hljs-title class_">IO</span>.puts(<span class="hljs-string">&quot;Hello, world!&quot;</span>)

marked.use(markedHighlight({...}));
marked.parse(code);
// &lt;span class=<span class="hljs-string">&quot;hljs-title class_&quot;</span>&gt;<span class="hljs-title class_">IO</span>&lt;/span&gt;.puts(&lt;span class=<span class="hljs-string">&quot;hljs-string&quot;</span>&gt;&amp;quot;<span class="hljs-title class_">Hello</span>, world!&amp;quot;&lt;/span&gt;)

The easiest way to simulate this behavior:

import hljs from 'highlight.js';
import * as marked from 'marked';
import { markedHighlight } from 'marked-highlight';

const extension = markedHighlight({
  langPrefix: 'hljs language-',
  highlight(code, lang) {
    const language = hljs.getLanguage(lang) ? lang : 'plaintext';
    return hljs.highlight(code, { language }).value;
  }
});

marked.use(extension);
marked.use(extension);
marked.use(extension);

marked.parse(code);

I believe that most modern reactive frameworks like React and Svelte reported here, have this tendency of calling things multiple times. Just using the same component twice on the same page, or hopping from one page to another in a Single Page Application, could trigger this accumulation of extensions because of the use behavior.

So, my suggestion would be to tweak this approach to make it more compatible with modern frameworks. I think we're likely to see more of this type of issue in the future.

Here are some potential approaches that come to mind:

We could mention explicitly in the documentation that invoking use twice will register your extension twice, leading to its invocation twice over the same content recursively. However, I don't see this as a viable long-term solution. It might prompt people to devise various workarounds to manage this in reactive applications, such as developing a kind of singleton or global control to ensure use is registered only once.
We could make marked instantiable. Currently, marked is global, meaning if two areas import it and set different extensions for the same task, but with varying configurations, both will be invoked, creating conflicting outputs. I think this could be a solid long-term approach. It provides isolation and flexibility for utilizing different instances of marked in the same application, configured differently for distinct purposes. Implementing this, while ensuring backward compatibility, may be a challenge.
We could provide a mechanism for users to easily identify if an extension is already registered (globally). Could we offer identifiers to extensions to prevent them from being added more than once? For example:

marked.use('extension-a', highlightExtension);
marked.use('extension-b', highlightExtension);

Or:

marked.use({...highlightExtension, ...{id: 'extension-a'}});
marked.use({...highlightExtension, ...{id: 'extension-b'}});

Thoughts?

Tony Brix · Answer 7 · Wed May 31 2023 07:04:23 GMT+0800 (China Standard Time)

See markedjs/marked#907

icebaker · Answer 8 · Wed May 31 2023 20:58:44 GMT+0800 (China Standard Time)

Thanks for sharing; this is great context!

So, for those struggling to get this working with reactive frameworks, the simple workaround is just to encapsulate marked into a singleton, and you're good to go:

import * as marked from 'marked';
import { markedHighlight } from 'marked-highlight';
import hljs from 'highlight.js';

const Marked = {
  instantiated: false,

  setup: () => {
    marked.setOptions({ mangle: false, headerIds: false });

    marked.use(
      markedHighlight({
        langPrefix: 'hljs language-',
        highlight(code, lang) {
          const language = hljs.getLanguage(lang) ? lang : 'plaintext';
          return hljs.highlight(code, { language }).value;
        }
      })
    );

    Marked.instantiated = true;
  },

  instance: () => {
    if (!Marked.instantiated) Marked.setup();

    return marked;
  }
};

export default Marked;

And then:

import Marked from 'Marked';

const html = Marked.instance().parse(code);

Here's the "fix" in the demo repo: commit d59f1a6

Tony Brix · Answer 9 · Wed May 31 2023 21:03:36 GMT+0800 (China Standard Time)

Ya or use marked.setOptions(marked.getDefaults()) to reset the options at the beginning.

Mai Tri Toan · Answer 10 · Wed Jun 07 2023 17:46:27 GMT+0800 (China Standard Time)

yah, first render is correct, but rerender is not because marked-highlight decodes the part of the code that has already been decrypted. LoL

my crazy solution. This not working if markdown has class 'hljs'

marked.use(
  markedHighlight({
    langPrefix: 'hljs language-',
    highlight(code, lang) {
      if(code.includes('class="hljs') return // 🤣🤣🤣 that crazy
      const language = hljs.getLanguage(lang) ? lang : 'plaintext';
      return hljs.highlight(code, { language }).value;
    }
  })
);

Tony Brix · Answer 11 · Wed Jun 07 2023 21:25:14 GMT+0800 (China Standard Time)

This should be fixed with markedjs/marked#2831

Tony Brix · Answer 12 · Sat Jun 10 2023 11:36:28 GMT+0800 (China Standard Time)

Should be fixed in marked v5.1.0