`wasm-pack` should generate isomorphic code
loynoir opened this issue Β· comments
π‘ Feature description
wasm-pack
should generate isomorphic code
π» Basic example
Below two targets have different output.
$ cd hello-world
$ wasm-pack build -t nodejs
$ wasm-pack build -t web
Differences are
-
node don't have init, while browser have init
-
node require
TextDecoder
from util, while browser check global -
node use
readFileSync
, while browser use fetch -
node generate CJS, while browser generate ESM
But,
-
can embed, instead of init
-
Quote https://nodejs.org/api/util.html,
The TextDecoder class is also available on the global object
-
And nodejs now support ESM for a long time. And now maintenance LTS is node16. Should drop CJS.
-
I think, this should be implement within
wasm-pack
, instead ofwasm-bindgen
So,
- I suggest,
wasm-pack
should embed the wasm generated bywasm-bindgen
as base64 string, and generate same ESM code for node and web
Related
Please π
Bump. This would be incredible to have. Is anyone on the team working on this? Can we support in anyway? π«‘
Isomorphic code would be amazing.
Workaround, tested same code works on both node and browser.
wasm-pack build --target nodejs --out-dir ./dist/pkg && node ./patch.mjs
import { readFile, writeFile } from "node:fs/promises";
const name = "xxx";
const content = await readFile(`./dist/pkg/${name}.js`, "utf8");
const patched = content
// use global TextDecoder TextEncoder
.replace("require(`util`)", "globalThis")
// inline bytes Uint8Array
.replace(
/\nconst path.*\nconst bytes.*\n/,
`
var __toBinary = /* @__PURE__ */ (() => {
var table = new Uint8Array(128);
for (var i = 0; i < 64; i++)
table[i < 26 ? i + 65 : i < 52 ? i + 71 : i < 62 ? i - 4 : i * 4 - 205] = i;
return (base64) => {
var n = base64.length, bytes = new Uint8Array((n - (base64[n - 1] == "=") - (base64[n - 2] == "=")) * 3 / 4 | 0);
for (var i2 = 0, j = 0; i2 < n; ) {
var c0 = table[base64.charCodeAt(i2++)], c1 = table[base64.charCodeAt(i2++)];
var c2 = table[base64.charCodeAt(i2++)], c3 = table[base64.charCodeAt(i2++)];
bytes[j++] = c0 << 2 | c1 >> 4;
bytes[j++] = c1 << 4 | c2 >> 2;
bytes[j++] = c2 << 6 | c3;
}
return bytes;
};
})();
const bytes = __toBinary(${JSON.stringify(await readFile(`./dist/pkg/${name}_bg.wasm`, "base64"))
});
`,
);
// deal with `imports['__wbindgen_placeholder__']`
// TODO: optimize with `__wbg_get_imports`
const wrapped = `export default (function() {
const module = { exports: {} };
${patched}
return module.exports;
})()
`;
await writeFile(`./dist/${name}.mjs`, wrapped);
@loynoir Awesome work! I love how you made it isomorphic. Previously, I was doing something similar, inlining base64 to bring wasm-pack libraries (web target) into a web worker within a SvelteKit project (because compiled SvelteKit web worker has difficulty in fetching wasm file unless inlined). I didn't think it could be extended this far, so kudos for that! π€©
I have a quick question though. Your approach allows importing the entire wasm-pack lib like so:
import wasm from "my-wasm-lib";
const { add } = wasm;
console.log(add(1, 2))
Would it be feasible to import individual functions directly? Something akin to:
import { add } from "my-wasm-lib";
console.log(add(1, 2))
import { readFile, writeFile } from "node:fs/promises";
const cargoTomlContent = await readFile("./Cargo.toml", "utf8");
const cargoPackageName = /\[package\]\nname = "(.*?)"/.exec(cargoTomlContent)[1]
const name = cargoPackageName.replace(/-/g, '_')
const content = await readFile(`./dist/pkg/${name}.js`, "utf8");
const patched = content
// use global TextDecoder TextEncoder
.replace("require(`util`)", "globalThis")
// attach to `imports` instead of module.exports
.replace("= module.exports", "= imports")
.replace(/\nmodule\.exports\.(.*?)\s+/g, "\nexport const $1 = imports.$1 ")
.replace(/$/, 'export default imports')
// inline bytes Uint8Array
.replace(
/\nconst path.*\nconst bytes.*\n/,
`
var __toBinary = /* @__PURE__ */ (() => {
var table = new Uint8Array(128);
for (var i = 0; i < 64; i++)
table[i < 26 ? i + 65 : i < 52 ? i + 71 : i < 62 ? i - 4 : i * 4 - 205] = i;
return (base64) => {
var n = base64.length, bytes = new Uint8Array((n - (base64[n - 1] == "=") - (base64[n - 2] == "=")) * 3 / 4 | 0);
for (var i2 = 0, j = 0; i2 < n; ) {
var c0 = table[base64.charCodeAt(i2++)], c1 = table[base64.charCodeAt(i2++)];
var c2 = table[base64.charCodeAt(i2++)], c3 = table[base64.charCodeAt(i2++)];
bytes[j++] = c0 << 2 | c1 >> 4;
bytes[j++] = c1 << 4 | c2 >> 2;
bytes[j++] = c2 << 6 | c3;
}
return bytes;
};
})();
const bytes = __toBinary(${JSON.stringify(await readFile(`./dist/pkg/${name}_bg.wasm`, "base64"))
});
`,
);
await writeFile(`./dist/${name}.mjs`, patched);
@loynoir Works brilliantly! Thanks!