wooorm / refractor

Just started using Refractor in a new project and it's been working really well. We have implemented it as follows to load languages dynamically:

const matches = codeBlockElement.match('language-([a-z]*)');
const language = matches && matches[1];
const code = codeElement.innerText;

let refractorNode;

/**
 * Try to load the lang, register it, and render the code block with Refractor.
 * If the lang is not recognized, just do nothing and we'll render it manually below.
 * */
try {
  // eslint-disable-next-line import/no-dynamic-require
  const requiredLanguage = require(`refractor/lang/${language}`);

  if (requiredLanguage && !Refractor.hasLanguage(language)) {
    Refractor.registerLanguage(requiredLanguage);
  }

  refractorNode = <Refractor language={language} value={code} />;
} catch {
  // Do nothing
}

return (
  {refractorNode || (
    <pre>
      <code className={`${language} language-${language || 'unknown'}`}>{code}</code>
    </pre>
  )}
);

Assume here that codeBlockElement is a DOM <code> element. This code executes after rendering markdown to HTML, in between when the HTML is injected into the virtual DOM (using Interweave)

So far so good... until we try and render an ERB block (indenting so you can see the backticks):

    ```erb
    <%= @foo %>
    ```

With that markdown, the language variable above gets set to 'erb' and the code variable gets set to '<%= @foo %>'. The whole block above actually executes without error but then this error is thrown after:

  15 | })
  16 | Prism.hooks.add('before-tokenize', function(env) {
  17 |   var erbPattern = /<%=?[\s\S]+?%>/g
> 18 |   Prism.languages['markup-templating'].buildPlaceholders(
  19 | ^    env,
  20 |     'erb',
  21 |     erbPattern

I looked at #9, which seems to be related. I will try to add a failing spec in the meantime.

Thanks for your help in advance.

Prism expects everything to be registered, and has no concept of dependencies. That’s why we build our files (see how) into forks with babel.

Our ERB fork does require markup-templating here:

refractor/lang/erb.js

Line 2 in 716fe90

var refractorMarkupTemplating = require('./markup-templating.js')

Could it be that the way you’re dynamically loading syntaxes does not account for sub dependencies?

Yeah, totally. Is there any penalty to always require('./markup-templating.js')?

After more consideration though... isn't my (albeit dynamic) require('refractor/lang/erb') enough to subsequently require('./markup-templating.js')?

Updating my code where I load the language to this seems to work:

  if (requiredLanguage && !Refractor.hasLanguage(language)) {
    Refractor.registerLanguage(requiredLanguage);
    Refractor.registerLanguage(require('refractor/lang/markup-templating.js'));
  }

This still smells like a bug though to me. Requiring a language should load all its dependencies, right?

Say we have the following index.js:

var refractor = require('refractor/core')
var erb = require('refractor/lang/erb')

refractor.register(erb)

console.log(refractor.highlight('something', 'erb'))

And the package.json looks as follows:

{
  "name": "example",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "dependencies": {
    "refractor": "^2.9.0"
  },
  "devDependencies": {
    "browserify": "^16.2.3"
  },
  "scripts": {
    "test": "browserify . -o build.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Now, if we run npm test, we get a build.js file. If you include that file in a browser for example, there are no errors and it works as expected.

This leads me to believe there are no problems with refractor, and therefor it’s not an issue I can solve, but it has to do with the way you’re loading the code!

Yes, probably. It looks like I'm never doing this:

var refractor = require('refractor/core')

Instead, I'm doing this:

import Refractor from 'react-refractor';

Then I'm calling Refractor.registerLanguage(...) instead of refractor.register(...). Can you explain the difference to me?

I'm sorry but I don't know the exact difference, as I haven't used the react project! 🤷‍♂️ maybe you can ask them?

TypeError: Cannot read property 'buildPlaceholders' of undefined