feat: support compiling to a Web Assembly (WASM) module
HarikrishnanBalagopal opened this issue · comments
Feature request
Support compiling to web assembly so that it can be used embedded in a web page.
This feature is already supported by the Golang implementation of Starlark:
https://github.com/google/starlark-go
https://haribala.dev/starlark-webasm-demo
Environment
MacOS Monterey 12.5.1 (21G83) (Intel CPU)
$ rustup toolchain list
stable-x86_64-apple-darwin
nightly-x86_64-apple-darwin (default) (override)
Actual behaviour
Currently there are lots of errors, mostly due to dependencies on stuff like use std::os::unix::io::AsRawFd;
$ cargo build --target wasm32-unknown-unknown
Compiling memchr v2.5.0
Compiling serde v1.0.145
Compiling cfg-if v1.0.0
Compiling unicode-width v0.1.10
Compiling tinyvec_macros v0.1.0
Compiling serde_json v1.0.85
Compiling ryu v1.0.11
Compiling itoa v1.0.3
Compiling bitflags v1.3.2
Compiling crossbeam-utils v0.8.12
Compiling num-traits v0.2.15
Compiling ahash v0.7.6
Compiling log v0.4.17
Compiling percent-encoding v2.2.0
Compiling unicode-bidi v0.3.8
Compiling smallvec v1.9.0
Compiling tinyvec v1.6.0
Compiling once_cell v1.15.0
Compiling textwrap v0.11.0
Compiling form_urlencoded v1.1.0
Compiling nibble_vec v0.1.0
Compiling num-integer v0.1.45
Compiling unicode-linebreak v0.1.4
Compiling aho-corasick v0.7.19
Compiling libc v0.2.134
Compiling regex-syntax v0.6.27
Compiling atty v0.2.14
Compiling endian-type v0.1.2
Compiling vec_map v0.8.2
Compiling ansi_term v0.12.1
Compiling bit-vec v0.6.3
Compiling strsim v0.8.0
Compiling dirs-sys-next v0.1.2
Compiling crossbeam-channel v0.5.6
Compiling dirs-next v2.0.0
Compiling radix_trie v0.2.1
Compiling bit-set v0.5.3
Compiling clap v2.34.0
Compiling thiserror v1.0.37
Compiling hashbrown v0.12.3
Compiling anyhow v1.0.65
Compiling inventory v0.1.11
Compiling erased-serde v0.3.23
Compiling memoffset v0.6.5
Compiling num-bigint v0.4.3
Compiling os_str_bytes v6.3.0
Compiling fd-lock v3.0.6
Compiling gazebo v0.8.0
error[E0255]: the name `unsupported` is defined multiple times
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/mod.rs:15:17
|
14 | mod unsupported;
| ---------------- previous definition of the module `unsupported` here
15 | pub use unsupported;
| ^^^^^^^^^^^ `unsupported` reimported here
|
= note: `unsupported` must be defined only once in the type namespace of this module
help: you can use `as` to change the binding name of the import
|
15 | pub use unsupported as other_unsupported;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error[E0433]: failed to resolve: could not find `unix` in `os`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/unsupported/read_guard.rs:2:14
|
2 | use std::os::unix::io::AsRawFd;
| ^^^^ could not find `unix` in `os`
error[E0433]: failed to resolve: could not find `unix` in `os`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/unsupported/rw_lock.rs:2:14
|
2 | use std::os::unix::io::AsRawFd;
| ^^^^ could not find `unix` in `os`
error[E0433]: failed to resolve: could not find `unix` in `os`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/unsupported/write_guard.rs:2:14
|
2 | use std::os::unix::io::AsRawFd;
| ^^^^ could not find `unix` in `os`
error[E0365]: `unsupported` is private, and cannot be re-exported
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/sys/mod.rs:15:17
|
15 | pub use unsupported;
| ^^^^^^^^^^^ re-export of private `unsupported`
|
= note: consider declaring type or module `unsupported` with `pub`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:20:43
|
20 | pub struct RwLockReadGuard<'lock, T: sys::AsRaw> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockReadGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:21:17
|
21 | guard: sys::RwLockReadGuard<'lock, T>,
| ^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockReadGuard;
|
1 | use std::sync::RwLockReadGuard;
|
help: if you import `RwLockReadGuard`, refer to it directly
|
21 - guard: sys::RwLockReadGuard<'lock, T>,
21 + guard: RwLockReadGuard<'lock, T>,
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:24:21
|
24 | impl<'lock, T: sys::AsRaw> RwLockReadGuard<'lock, T> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockReadGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:25:35
|
25 | pub(crate) fn new(guard: sys::RwLockReadGuard<'lock, T>) -> Self {
| ^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockReadGuard;
|
1 | use std::sync::RwLockReadGuard;
|
help: if you import `RwLockReadGuard`, refer to it directly
|
25 - pub(crate) fn new(guard: sys::RwLockReadGuard<'lock, T>) -> Self {
25 + pub(crate) fn new(guard: RwLockReadGuard<'lock, T>) -> Self {
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:30:14
|
30 | impl<T: sys::AsRaw> ops::Deref for RwLockReadGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/read_guard.rs:40:14
|
40 | impl<T: sys::AsRaw> Drop for RwLockReadGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:13:27
|
13 | pub struct RwLock<T: sys::AsRaw> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLock` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:14:16
|
14 | lock: sys::RwLock<T>,
| ^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLock;
|
1 | use std::sync::RwLock;
|
help: if you import `RwLock`, refer to it directly
|
14 - lock: sys::RwLock<T>,
14 + lock: RwLock<T>,
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:17:14
|
17 | impl<T: sys::AsRaw> RwLock<T> {
| ^^^^^ not found in `sys`
error[E0433]: failed to resolve: could not find `RwLock` in `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/rw_lock.rs:34:24
|
34 | lock: sys::RwLock::new(inner),
| ^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLock;
|
1 | use std::sync::RwLock;
|
help: if you import `RwLock`, refer to it directly
|
34 - lock: sys::RwLock::new(inner),
34 + lock: RwLock::new(inner),
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:20:44
|
20 | pub struct RwLockWriteGuard<'lock, T: sys::AsRaw> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockWriteGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:21:17
|
21 | guard: sys::RwLockWriteGuard<'lock, T>,
| ^^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockWriteGuard;
|
1 | use std::sync::RwLockWriteGuard;
|
help: if you import `RwLockWriteGuard`, refer to it directly
|
21 - guard: sys::RwLockWriteGuard<'lock, T>,
21 + guard: RwLockWriteGuard<'lock, T>,
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:24:21
|
24 | impl<'lock, T: sys::AsRaw> RwLockWriteGuard<'lock, T> {
| ^^^^^ not found in `sys`
error[E0412]: cannot find type `RwLockWriteGuard` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:25:35
|
25 | pub(crate) fn new(guard: sys::RwLockWriteGuard<'lock, T>) -> Self {
| ^^^^^^^^^^^^^^^^ not found in `sys`
|
help: consider importing one of these items
|
1 | use crate::RwLockWriteGuard;
|
1 | use std::sync::RwLockWriteGuard;
|
help: if you import `RwLockWriteGuard`, refer to it directly
|
25 - pub(crate) fn new(guard: sys::RwLockWriteGuard<'lock, T>) -> Self {
25 + pub(crate) fn new(guard: RwLockWriteGuard<'lock, T>) -> Self {
|
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:30:14
|
30 | impl<T: sys::AsRaw> ops::Deref for RwLockWriteGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:39:14
|
39 | impl<T: sys::AsRaw> ops::DerefMut for RwLockWriteGuard<'_, T> {
| ^^^^^ not found in `sys`
error[E0405]: cannot find trait `AsRaw` in module `sys`
--> /Users/user/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-lock-3.0.6/src/write_guard.rs:47:14
|
47 | impl<T: sys::AsRaw> Drop for RwLockWriteGuard<'_, T> {
| ^^^^^ not found in `sys`
Compiling unicode-normalization v0.1.22
Compiling fnv v1.0.7
Some errors have detailed explanations: E0255, E0365, E0405, E0412, E0433.
For more information about an error, try `rustc --explain E0255`.
error: could not compile `fd-lock` due to 22 previous errors
warning: build failed, waiting for other jobs to finish...
$ echo $?
101
What is recommended way to test for webassembly code these days?
Last time I tried to make starlark-rust work in webassembly I have not find a good way to do it. For example, when test failed, no stack trace was printed (if anything printed at all, sometimes it just crashed without any message). I tried different wasm interpreters, none were good enough.
I think a blocker for this is also #44 since wasm is only 32 bit. I got it compiling after commenting out a lot of type size assertions, but definitely doesn't work because of tagged pointers, etc.
@stepancheg Not sure if this helps, but I pushed a branch where you can repro it erroring (also using bazel). I pushed a branch here
If you install bazel & nodejs you can probably repro a crash like
bazel build //starlark_wasm:starlark_wasm_bindgen_nodejs && echo "(async () => await import('./bazel-bin/starlark_wasm/starlark_wasm_bindgen_nodejs.js'))()" | nodejs --experimental-wasm-modules -
it fails similarly in the browser too (i have a diff target set up for it).
I get this output
echo "(async () => await import('./bazel-bin/starlark_wasm/starlark_wasm_bindgen_nodejs.js'))()" | nodejs --experimental-wasm-modules -
Hello from Rust!
Cheese 2.0
ast parsed
evaluator created
globals created
wasm://wasm/045a6832:1
RuntimeError: unreachable
at __rust_start_panic (wasm://wasm/045a6832:wasm-function[44297]:0xae737f)
at rust_panic (wasm://wasm/045a6832:wasm-function[35606]:0xabe33b)
at std::panicking::rust_panic_with_hook::hd000e9fb43b5781d (wasm://wasm/045a6832:wasm-function[17236]:0x8b75f2)
at std::panicking::begin_panic_handler::{{closure}}::he16e52e9a7dddeb1 (wasm://wasm/045a6832:wasm-function[21401]:0x979991)
at std::sys_common::backtrace::__rust_end_short_backtrace::h227361e053771d9e (wasm://wasm/045a6832:wasm-function[34839]:0xab6332)
at rust_begin_unwind (wasm://wasm/045a6832:wasm-function[29593]:0xa60e85)
at core::panicking::panic_fmt::h9d972fcdb087ce21 (wasm://wasm/045a6832:wasm-function[33731]:0xaa79a2)
at starlark::values::layout::heap::arena::Arena::alloc_uninit::hbeaf5c51c0d56424 (wasm://wasm/045a6832:wasm-function[1100]:0x38955b)
at starlark::values::layout::heap::arena::Arena::alloc::h7d9ce7ed4dcd4283 (wasm://wasm/045a6832:wasm-function[3131]:0x5203fe)
at starlark::values::layout::heap::heap_type::FrozenHeap::alloc_raw::h2f4db6bad7ed0b29 (wasm://wasm/045a6832:wasm-function[30180]:0xa6caf0)
I had to modify some other things just to get it to compile (this is also my first time dabbling w/ rust or wasm so I probably broke stuff with my changes)
Why Bazel? Can't it be built with cargo?
I think it could be built in cargo (pretty sure it would possibly be easier), but was mostly working with an existing project had bazel.
wasm32-wasi works since e0c1d81.
wasm32-unknown-unknown probably doesn't work.
A demo web page would be awesome!
Closing since this works.