Use with webpack?

[Paul Razvan Berg]

This is a great library, but I struggle to make it work with webpack 4. This is the rule I'm applying:

{
    test: /\.wasm$/,
    exclude: /node_modules/,
    type: 'webassembly/experimental',
}

(Even if I don't think it's necessary as per their official example)

When I try to use the generated bundle in a nodejs project, I get the following error when using the development build:

Error: ENOENT: no such file or directory, open '/mcl_c.wasm'

And this error when using the production build:

On the web, we need the wasm binary to be preloaded and set on Module['wasmBinary']. emcc.py will do that for you when generating HTML (but not JS)

I tried to put that file in the same folder as the bundle, in my nodejs projects, literally everywhere, but nothing worked. Is it that some specific configuration of this library makes it impossible to use it with webpack?

Potentially related: webpack/webpack#7352

[Paul Razvan Berg]

It seems the development build error was caused by this: webpack/webpack#1599

Fixed that, but still stuck on the other issue.

[Junpei Kawamoto]

Putting .wasm files on the document root like this works for me:

<document root>
├── index.html
├── lib
│   └── vendor.js
├── mcl_c.wasm
└── mcl_c512.wasm

This library looks like it uses neither import nor require to load wasm files but fetching them from the root. I think webpack couldn't know if those wasm files should be bundled and we need to prepare them by ourselves.

Btw, this is a part of my webpack.config.js and you'd need to exclude mcl-wasm/test.js explicitly because it requires perf_hooks but browsers don't have it.

module: {
  noParse: /mcl-wasm\/test\.js/,
  rules: [
    {
      test: /\.jsx?$/,
      exclude: /node_modules/,
      use: [
        {
          loader: "babel-loader",
        },
      ],
    },
  ]
} 

[taoxinyi]

One workaround is to change var wasmBinaryFile="mcl_c.wasm" in mcl_c.js (Similar in mcl_c512.js) to whatever path you want to. By default it will get from the root, so put *.wasm in the root will be fine.

[Paul Razvan Berg]

My goal is to somehow import the wasm file into the JavaScript bundle itself, so that anyone using our library doesn't have to copy and paste the wasm binary files. Until further notice, we'll just avoid importing wasm in the web target of webpack.

Specifically, these are the issues I encountered:

• webpack/webpack#7388
• webpack/webpack#7352

I tried to follow the webpack example, but that didn't get me anywhere with this library. Ostensibly, the problem is that webpack doesn't work well with Emscripten (which I guess it's what has been used here?).

[Paul Razvan Berg]

Also, this might be a silly note, but I'm not fully sure I understand what this block of code does:

fetch(`./${name}.wasm`) // eslint-disable-line
    .then((response) => response.arrayBuffer())
    .then((buffer) => new Uint8Array(buffer))
    .then(() => {
        exports.mod = Module(); // eslint-disable-line
        exports.mod.cryptoGetRandomValues = _cryptoGetRandomValues;
        exports.mod.onRuntimeInitialized = () => {
            setup(exports, curveType);
            resolve();
        };
    });

Obviously, it's trying to fetch the file, but I don't see the output of that being used. Is it that WebAssembly handles the wasm file when creating a new object via the Module() constructor?

[MITSUNARI Shigeo]

@PaulRBerg
The code block is for browser.

I do not know webpack and so how to support it yet.
I'll investigate it.

[Junpei Kawamoto]

Bundling all files including WASM files into one file sounds nice. I tried a couple of things for it.

When I imported mcl_c.wasm with this rule

{
  test: /\.wasm$/,
  type: “webassembly/experimental”,
}

I got webpack/webpack#7388. Even when I imported it with raw-loader, I got this error

ERROR in ./node_modules/mcl-wasm/mcl_c.wasm
Module parse failed: magic header not detected
You may need an appropriate loader to handle this file type.

It looks like webpack expects to load the WASM file with type: “webassembly/experimental” but it doesn’t work for us.

Only one hacky workaround I found is embedding the WASM file as an image file to trick webpack. If mcl_c.wasm is renamed mcl_c.png and imported by require(“./mcl_c.png”); with raw-loader

{
  test: /\.png$/,
  loader: “raw-loader”
}

webpack doesn’t raise any errors.

The imported data are now base64 encoded and we need to decode them. In my case, the original WASM file is loaded like this in mcl_c.js:

WebAssembly.instantiateStreaming(
  fetch(wasmBinaryFile, { credentials: “same-origin” }),
  info
)

so I replaced it like this:

const wasm = require("./mcl_c.png");
const bin = atob(wasm.replace(/^.*,/, ""));
const buf = new Uint8Array(bin.length);
for (let i = 0; i < bin.length; i++) {
  buf[i] = bin.charCodeAt(i);
}
WebAssembly.instantiateStreaming(
  Promise.resolve(new Response(new Blob([buf.buffer]), {
    headers: {
      "content-type": "application/wasm"
    }
  })),
  info
)

There would be a better way for sure (perhaps webpack/webpack#7352 (comment)), but it works for me so far.

[MITSUNARI Shigeo]

I've merged wasm file into mcl.js at the latest version.