Giters

ktock / container2wasm

Container to WASM converter

Home Page:https://ktock.github.io/container2wasm-demo/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Not working with image created by Nix

vaibhavsagar opened this issue · comments

commented

Hi, I can't seem to get this working when I try to convert a container image built by Nix to WASM. Here's what my Nix expression looks like:

# default.nix
let
  nixpkgs-src = builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/tarball/7731670498f0a22c361c9d68f73d382bce05d7dc";
    sha256 = "sha256-M59UCWsadEFO+DMdZFMtTlxpFyT3RQIpBLuEX8oziVc=";
  };
  nixpkgs = import nixpkgs-src {};
  haskellEnv = nixpkgs.haskellPackages.ghcWithPackages (p: [ ]);
  image = nixpkgs.dockerTools.buildLayeredImage {
    name = "small-haskell";
    tag = "latest";
    contents =  [
      haskellEnv
      nixpkgs.bashInteractive
    ];
    config = {
      Cmd = ["${haskellEnv}/bin/ghci"];
    };
  };
in image

And here are the commands I'm running:

$ nix-build default.nix
$ docker load < result
$ c2w --version
c2w version v0.6.2
$ c2w small-haskell:latest out.wasm
<...>
#120 [bochs-dev-packed 1/2] RUN /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out
#120 128.3 panicked at 'capacity overflow', library/alloc/src/raw_vec.rs:517:5
#120 128.4 thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: the `wasi_vfs_pack_fs` function trapped
#120 128.4 
#120 128.4 Caused by:
#120 128.4     wasm trap: wasm `unreachable` instruction executed
#120 128.4     wasm backtrace:
#120 128.4         0: 0x57515 - <unknown>!<wasm function 540>
#120 128.4         1: 0x5aa96 - <unknown>!<wasm function 567>
#120 128.4         2: 0x5880b - <unknown>!<wasm function 549>
#120 128.4         3: 0x2e180 - <unknown>!<wasm function 229>
#120 128.4         4: 0x3688d - <unknown>!<wasm function 293>
#120 128.4         5: 0x3ce1a - <unknown>!<wasm function 330>
#120 128.4         6: 0x513aa - <unknown>!<wasm function 497>
#120 128.4     ', crates/wasi-vfs-cli/src/bin/wasi-vfs.rs:5:32
#120 128.4 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
#120 ERROR: process "/bin/sh -c /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out" did not complete successfully: exit code: 101
------
 > [bochs-dev-packed 1/2] RUN /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out:
128.4     wasm backtrace:
128.4         0: 0x57515 - <unknown>!<wasm function 540>
128.4         1: 0x5aa96 - <unknown>!<wasm function 567>
128.4         2: 0x5880b - <unknown>!<wasm function 549>
128.4         3: 0x2e180 - <unknown>!<wasm function 229>
128.4         4: 0x3688d - <unknown>!<wasm function 293>
128.4         5: 0x3ce1a - <unknown>!<wasm function 330>
128.4         6: 0x513aa - <unknown>!<wasm function 497>
128.4     ', crates/wasi-vfs-cli/src/bin/wasi-vfs.rs:5:32
128.4 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
------
container2wasm1665798681:488
--------------------
 486 |     
 487 |     FROM bochs-dev-${OPTIMIZATION_MODE} AS bochs-dev-packed
 488 | >>> RUN /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out
 489 |     ARG OUTPUT_NAME
 490 |     RUN mv packed /out/$OUTPUT_NAME
--------------------
ERROR: failed to solve: process "/bin/sh -c /tools/wasi-vfs/wasi-vfs pack /Bochs/bochs/bochs --mapdir /pack::/minpack -o packed && mkdir /out" did not complete successfully: exit code: 101
exit status 1

Am I using the tool incorrectly?

commented

@vaibhavsagar Thanks for reporting this.

#120 128.3 panicked at 'capacity overflow', library/alloc/src/raw_vec.rs:517:5

Seems to related to memory? Could you check if it causes out of memory?

commented

Thanks for your response! Where do you do think I'm running out of memory, is it the Docker daemon, and how would I check this? The computer I'm running this on has 32GB of RAM installed which I think should be enough?

commented

I ran it again while watching htop in a separate window and I got the same error even though total system RAM usage did not go above 3GB. I tried having docker stats open as well but weirdly did not see anything.

commented

It seems that wasi-vfs fails to package large files that are over several GB. I keep on investigating the root cause but, as a workaround, you can avoid the failure by using a smaller image. (On my machine, small-haskell is 2.79GB. Could you create smaller image if possible?)

commented

Thanks for investigating! I also tried with haskell:slim-buster and got the same error, even though it's a little smaller (2.28GB). Do you know what the size limit is?