wasm-opt: Exported global cannot be mutable

Question

wasm-opt: Exported global cannot be mutable

Urhengulas opened this issue 4 years ago · comments

🐛 Bug description

I am using wasm-pack and wasm-bingen to build a javascript package with rust.

Since today when I am running wasm-pack build I am getting following error:

[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
    Finished release [optimized + debuginfo] target(s) in 13.59s
[INFO]: Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[wasm-validator error in module] unexpected true: Exported global cannot be mutable, on 
global$0

# --- omitted .wast output ---

Fatal: error in validating input
Error: failed to execute `wasm-opt`: exited with exit code: 1
  full command: "~/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt" "$REPO/rav1e_js/pkg/rav1e_js_bg.wasm" "-o" "$REPO/rav1e_js/pkg/rav1e_js_bg.wasm-opt.wasm" "-O"
To disable `wasm-opt`, add `wasm-opt = false` to your package metadata in your `Cargo.toml`.

It was still working on master yesterday (see CI), but doesn't anymore today with the same commit (see CI). (The important step is "Build").

🤔 Expected Behavior

Build package and optimize it with wasm-opt as it did before.

(Running wasm-pack build --dev works, but I'd like to have the optimization).

👟 Steps to reproduce

Trying a few things I figured out that I can make wasm-opt pass when I am removing all exported (non-static) methods on the exported structs (which is abviously not desireable).

On urhengulas/rav1e@wasm-opt-fix I removed all the methods and it builds and optimizes fine:

➜  rav1e git:(wasm-opt-fix) ✗ cd rav1e_js && wasm-pack build
[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
    Finished release [optimized + debuginfo] target(s) in 0.05s
[INFO]: License key is set in Cargo.toml but no LICENSE file(s) were found; Please add the LICENSE file(s) to your project directory
[INFO]: Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: :-) Done in 0.88s
[INFO]: :-) Your wasm pkg is ready to publish at /home/urhengulas/Documents/github.com/xiph/rav1e/rav1e_js/pkg.

But, as soon I am adding a method (e.g. fn Frame.debug(&self) -> Self in rav1e_js/src/frame.rs) back, I am getting the error from the bug description.

🌍 Your environment

Include the relevant details of your environment.

➜  rustc --version
rustc 1.45.0 (5c1f21c3b 2020-07-13)
➜  wasm-pack --version
wasm-pack 0.9.1
➜  ~/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt --version
wasm-opt version_90

Thx

Thanks to everyone taking the time and having a look at this issue!

Avery Harnish · Answer 1 · Thu Jul 30 2020 01:38:11 GMT+0800 (China Standard Time)

saw this same issue the other day in some e2e tests

Alixinne · Answer 2 · Thu Jul 30 2020 03:38:43 GMT+0800 (China Standard Time)

For me this issue showed up in CI. The last build that was correct before the failed one today was on July 21st. The nightly compiler from that version already exhibited the bug so it seems the cause is some dependency somewhere triggering this behavior which is not supported by wasm-opt, but I haven't been able to trace which one yet.

Johann Hemmann · Answer 3 · Thu Jul 30 2020 05:48:12 GMT+0800 (China Standard Time)

Thanks for the input @EverlastingBugstopper, @vtavernier.

I've created a bit more atomic example:

// src/lib.rs
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub struct Boat {
    length: u32,
}

#[wasm_bindgen]
impl Boat {
    // works
    pub fn new(length: u32) -> Self {
        Self { length }
    }

    // doesn't work
    pub fn float(&self) -> String {
        String::from("floating... ⛵")
    }

    // works
    pub fn volume(&self) -> u32 {
        self.length * 2
    }
}

// doesn't work
#[wasm_bindgen]
pub fn string() -> String {
    String::from("wasm-opt")
}

So it seems returning a String fails wasm-opt.

It is also failing, when returning some structs, but for others not. I will investigate what the failing ones have in common.

Alixinne · Answer 4 · Thu Jul 30 2020 06:04:19 GMT+0800 (China Standard Time)

For me the bug was introduced by wasm-bindgen 0.2.66 (0.2.65 is fine).

Johann Hemmann · Answer 5 · Thu Jul 30 2020 19:28:34 GMT+0800 (China Standard Time)

The problem seems to be with the non-MVP webassembly feature mutable-global. Adding --enable-mutable-globals to the wasm-opt command seems to fix it (see WebAssembly/binaryen#3006 (comment)).

My workaround is now:

disable wasm-opt in your Cargo.toml ...

[package.metadata.wasm-pack.profile.release]
wasm-opt = false

... run wasm-pack ...

$ wasm-pack build

... and optimize manually

$ /path/to/wasm-opt pkg/repo_bg.wasm -o pkg/repo_bg.wasm -O --enable-mutable-globals

(My /path/to/wasm-opt is ~/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt.)

Edit: Originally I recommended to use wasm-opt [...] -O -all, but this creates new errors.

Adam · Answer 6 · Sun Aug 02 2020 05:38:24 GMT+0800 (China Standard Time)

Ran into the same issue while doing the wasm game of life tutorial: https://rustwasm.github.io/docs/book/game-of-life/implementing.html

Deleted user · Answer 7 · Sun Aug 02 2020 20:45:32 GMT+0800 (China Standard Time)

@Urhengulas It's possible to specify wasm-opt arguments, so you should be able to do this:

[package.metadata.wasm-pack.profile.release]
wasm-opt = ["-Oz", "--enable-mutable-globals"]

Change -Oz to whatever optimization level you need.

Johann Hemmann · Answer 8 · Mon Aug 03 2020 18:43:28 GMT+0800 (China Standard Time)

@Urhengulas It's possible to specify wasm-opt arguments, so you should be able to do this:
[package.metadata.wasm-pack.profile.release]
wasm-opt = ["-Oz", "--enable-mutable-globals"]
Change -Oz to whatever optimization level you need.

Thanks @0xd4d. This surely is the better workaround.

Sven Sauleau · Answer 9 · Wed Aug 05 2020 22:04:00 GMT+0800 (China Standard Time)

Mutable globals are now phase 4 in the WebAssembly CG, it should be enabled by default in wasm-opt

Alon Zakai · Answer 10 · Wed Aug 05 2020 23:11:25 GMT+0800 (China Standard Time)

@xtuc I don't think we have a clear answer on that. Discussions have been going on about what the defaults should be for various tools including wasm-opt and LLVM and we should discuss those more on https://github.com/WebAssembly/tool-conventions cc @sbc100

The main risk I see is that we turn a feature on by default, but not all VMs support it yet, and if the developer doesn't test everywhere then they can end up shipping code that only works in some places.

Sam Clegg · Answer 11 · Wed Aug 05 2020 23:20:37 GMT+0800 (China Standard Time)

In this particular case I think we wasm-opt should be driven by the target-features section of the input file. So perhaps the problem is that llvm/wasm-ld doesn't know to add mutable-globals to the target-features of the output binary.

While I agree that using/adding new features during wasm-opt would be bad, I don't think limiting the input feature set is something wasm-opt should be doing. If my upstream toolchain is choosing to emit mutable-globals wasm-opt should honor that. The job of wasm-opt is not to limit features in its input, right?

Sven Sauleau · Answer 12 · Wed Aug 05 2020 23:35:31 GMT+0800 (China Standard Time)

My understanding of the target-features section is that it's used by the toolchain but doesn't necessarily reflect what the target browser supports. In JS we used to transpile down the unsupported features given some user preference.

Sam Clegg · Answer 13 · Wed Aug 05 2020 23:42:54 GMT+0800 (China Standard Time)

target-features is used by the toolchain to specific the feature set that the output is expecting. If llvm's output includes the use of given feature that feature should appear in the target-features section. If it doesn't, that is a bug IIUC.

If you want to transpile those features away in order to support older runtimes that seems fine too.

In this case it seems clear that this is the problem: llvm is producing code that depends a feature but that information not being correctly transmitted to wasm-opt.

Avery Harnish · Answer 14 · Thu Aug 06 2020 01:03:45 GMT+0800 (China Standard Time)

I don't have much context here on what exactly the problem is but I will say that whatever changed broke pretty much every standard Hello World project in the Rust-Wasm ecosystem and that doesn't feel great. Wherever this change was introduced needs to have some sort of integration test with some common tutorials so that releases don't break documentation across the entire ecosystem.

If a breaking change must happen (which I'd argue it probably doesn't) then there should be advance notice, some warning messages in terminal output before a breaking change (similar to how Rust itself deprecates things), and work put in to update documentation across the ecosystem that does not break with default settings.

edit: i think we might want to revisit whether wasm-pack should even run wasm-opt by default instead of having it be opt in until it has stabilized - these breaking changes are not fun to work with.

Jaco · Answer 15 · Thu Aug 06 2020 01:31:46 GMT+0800 (China Standard Time)

For me this ends up slightly more problematic - due to React Native support, I'm also compiling from the generated wasm to asm.js (same binaryen toolset, different binary), and cannot find a working option to make that work at all. (So -all works for wasm-opt and does seem to work on wasm2js, however the output fails all tests)

There is def. a mismatch with this tooling and the binaryen tooling that has "some" impacts.

TL;DR Can (based on suggestions here) get the wasm part compiled, bundled and optimized, however have no luck generating output to an asm.js bundle from the compiled WASM now being generated. For now fixed to the last know working .25 and all is well.

Johann Hemmann · Answer 16 · Fri Aug 07 2020 17:20:47 GMT+0800 (China Standard Time)

@EverlastingBugstopper: i think we might want to revisit whether wasm-pack should even run wasm-opt by default instead of having it be opt in until it has stabilized - these breaking changes are not fun to work with.

I agree. Maybe mirroring the cargo build behaviour would be an option. This is wasm-pack build builds in debug mode [what --dev currently does ([optimized + debuginfo], no wasm-opt)] and wasm-pack build --release builds the optimized wasm and also optimizes with wasm-opt (like it currently already does).

bperris · Answer 17 · Fri Aug 14 2020 02:18:50 GMT+0800 (China Standard Time)

I have been working on a work project which is now completely bombed by this issue. The workaround solves the issue here but it breaks something with wasm2js where __wbindex_export_2 no longer exists in the javascript output. We have to use wasm-opt because the debug code is way to big for our embedded environment that is running a javascript runtime.

I have reverted all the changes I made on the repo after making cargo dep updates / updates to binaryen binaries, and after doing a complete git clean -dxf which blows away all untracked objects, this issue won't go away now which wasn't an issue earlier today. It seems the issue lays in the tool cache that wasm-pack. I must of have had something older in there, but now it reports the same version as the OP here.

What can I do to revert this? I already had been using wasm-pack 0.9.1 before with no issues.

I cannot build this product app without either reverting completely or getting the patch that will resolve this issue completely.

Thanks, Bryan

Avery Harnish · Answer 18 · Fri Aug 14 2020 02:25:12 GMT+0800 (China Standard Time)

hey @bryanperris - wasm-pack will download binaries to your operating system cache location. if you install wasm-opt on your own i believe that wasm-pack will use that version instead (though i'm not 100% sure on that).

if you run RUST_LOG=debug wasm-pack build you should be able to see an info log that looks like this:

INFO 2020-08-13T18:23:05Z: wasm_pack::child: Running "~/Library/Caches/.wasm-pack/wasm-opt-a528729925722b63/wasm-opt" "~/Documents/work/_tmp/rusty/pkg/rusty_bg.wasm" "-o" "~/Documents/work/_tmp/rusty/pkg/rusty_bg.wasm-opt.wasm" "-O"

this should point you to the old cached version of wasm-opt that you can then remove. it's very possible that wasm-pack will just redownload it for you though... the project should probably pin wasm-opt to an older version that doesn't break everything but hopefully this helps with a workaround.

bperris · Answer 19 · Fri Aug 14 2020 02:32:29 GMT+0800 (China Standard Time)

I get the same error with wasm-opt used by cargo compilation and wasm2js
[wasm-validator error in module] unexpected true: Exported global cannot be mutable, on

For cargo compilation via webpack plugin: /home/bperris/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt
For wasm2js, we use an older version in our git repo, but that also generates the same error too.

I think my cache from months ago had downloaded tools that always worked for me, and today I somehow triggered them to be updated.

I would like to have wasm-pack just pin the cache to a certain version.

bperris · Answer 20 · Fri Aug 14 2020 03:02:24 GMT+0800 (China Standard Time)

version 79 works compiling our crate, but it doesn't work for asm2js.

Our repo contains binaryen version_93. I had a script calling wasm2js to generate the javascript but now it doesn't work anymore.

bperris · Answer 21 · Fri Aug 14 2020 03:20:34 GMT+0800 (China Standard Time)

if I just enable --enable-mutable-globals, I can get past crate compilation and wasm2js, but then I run into this issue:

I get a bunch of these warning messages from webpack.

WARNING in ./libproject/pkg/index_bg.js 764:12-36 "export '__wbindgen_export_2' (imported as 'wasm') was not found in './index_bg.wasm.js' @ ./libproject/pkg/index.js @ ./src/file.ts @ ./src/index.ts

bperris · Answer 22 · Wed Sep 09 2020 22:24:13 GMT+0800 (China Standard Time)

Fixed my issue by doing this: wasm-bindgen = "=0.2.60"

Other deps I use that depend on wasm-bindgen also must be downgraded, but they don't need the = in their versions.

Surma · Answer 23 · Thu Sep 17 2020 00:45:10 GMT+0800 (China Standard Time)

Fixed my issue by doing this: wasm-bindgen = "=0.2.60"

wasm-bindgen = "=0.2.65" is sufficient as per @vtavernier’s comment above. I am worried the compiler can emit exported mutable globals by default, so this might actually be an upstream rustc issue, not sure.

lukaslueg · Answer 24 · Thu Sep 17 2020 01:22:58 GMT+0800 (China Standard Time)

@surma rust-lang/rust#76404

Surma · Answer 25 · Thu Sep 17 2020 21:48:41 GMT+0800 (China Standard Time)

Thanks for that, @lukaslueg. I actually didn’t realized that exported mutable globals are now supported everywhere, so @0xd4d’s answer is actually the correct one and there’s no need to downgrade wasm-bindgen.

bperris · Answer 26 · Thu Sep 17 2020 22:14:53 GMT+0800 (China Standard Time)

It works for compiling to WebAssembly, but causes an issue for wasm2js.

bperris · Answer 27 · Fri Sep 18 2020 23:04:13 GMT+0800 (China Standard Time)

What I realized is that cargo always pulls updates but only at minor revisions only if the cargo.lock file doesn't exist and not using a restrictive version tag, like if you are using wasm-bindgen 0.2.62 (which I was using a few months ago), it will pull in the latest 0.2.xx wasm-bindgen crate. I think the minor change that introduced mutable-globals should of been a major change, but maybe not a concern to the wasm-bindgen folk since only binaryen has been affected by this (or really just wasm2js). The day I posted having this puke of debug output from binaryen must be when I blew away my cargo lock file.

This issue was hiding in the shadows for me since the release of 0.2.63, "The encoding and implementation of WebAssembly reference types has been sync'd with the latest upstream specification.".

Wojciech Niedźwiedź · Answer 28 · Thu Dec 10 2020 01:04:14 GMT+0800 (China Standard Time)

just stumbled upon this, what's the solution? update some dependency?

Zimon Tai · Answer 29 · Thu Dec 10 2020 10:23:28 GMT+0800 (China Standard Time)

@Niedzwiedzw currently my solution is to use a fork of wasm-pack, as it's not maintained now. Or you could directly use wasm-bindgen-cli to completely remove wasm-pack dependency.

Ari Lotter · Answer 30 · Tue Jan 05 2021 03:42:01 GMT+0800 (China Standard Time)

This issue may be fixed by rustwasm/wasm-bindgen#2391 , which prevents wasm-bindgen from exporting mutable globals for compatibility reasons.

Ari Lotter · Answer 31 · Thu Jan 28 2021 04:14:17 GMT+0800 (China Standard Time)

rustwasm/wasm-bindgen#2391 has been merged & released in wasm-bindgen 0.2.70. I believe this fixes the issue, and this ticket can now be closed.

Felipe Rocha · Answer 32 · Wed Aug 25 2021 09:52:11 GMT+0800 (China Standard Time)

Is this fixed? I just ran into it trying to compile the threads example: https://rustwasm.github.io/docs/wasm-bindgen/examples/raytrace.html