App using WasmEdge networking freezes when used with youki

[Gram]

WASI preview 1 does not implement some of the networking functions (like opening new sockets) required to implement a web server. Because of that, WasmEdge provides it's own extension for network sockets. See Available extensions.

I made a simple Go echo server using stealthrocket/net that relies on this API and it works with bare wasmedge. However, when running the same app with podman+youki+wasmedge, the app enters listening mode but never receives requests.

Steps to reproduce

Here is all the code:

youki-wasmedge.zip

1. Build youki with --with-wasmedge flag.
2. Install task
3. Run task
4. When you see listening... message, open a new terminal window and run task send. You'll see Failed to connect to localhost port 8000 after 0 ms: Connection refused
5. Now run task kill to kill the server (for some possibly related reason it doesn't respond to interrupt signals)
6. Run it instead with wasmedge run ./echo2.wasm.
7. Run task send again in another terminal. You'll see "HELLO" message as expected.

Why bother

It's a lot of steps, details, and technologies, and you may see no reason to investigate it. However, I believe there is a huge potential for WASM in the world of container images. With the help of TinyGo and youki, I made and ran in podman a cowsay CLI tool that produced just a 29 Kb image! And it starts twice faster than anything you could do with classic containers. See also #1278 (comment).

These two benefits make wasm containers a much better choice in many situations, like cloud functions. And most of such functions need this advanced networking. Use cases include consuming Redis or RabbitMQ messages, handling HTTP requests, or even providing Prometheus metrics.

[Toru Komatsu]

To be honest, I'm not 100% sure to support this issue. But this feature you suggest affects only relevant wasmedge code in youki, isn't this?
https://github.com/containers/youki/blob/main/crates/youki/src/workload/wasmedge.rs

If it is true, I agree with you. For now, we are in the phase to spread use cases wasm on Podman/Docker/K8s. So it would be great to add new wonderful features.

@orsinium Do you have a chance to contribute youki to implement it?

[Gram]

I tried to look into it already, yes. I know Rust but I don't know much about containers. I mean, I know how to use DOcker and stuff but I have no idea how in general youki works. Digging through libcontainer's Spec and Process and all that stuff I couldn't find anything related to publishing ports. How does publishing ports work for the default libcontainer executor?

[Toru Komatsu]

but I have no idea how in general youki works.

If you are not familiar with low-level container runtime, I recommend to read this docs:
https://containers.github.io/youki/user/basic_usage.html#using-youki-standalone

Digging through libcontainer's Spec and Process and all that stuff I couldn't find anything related to publishing ports. How does publishing ports work for the default libcontainer executor?

Low-level container runtimes aren't responsible for publishing ports for now. The role of low-level container runtime is defined by OCI Runtime Spec.
https://github.com/opencontainers/runtime-spec

[Yashodhan]

Hey, I looked into it, and if anything, the issue would lie in the wasmedge executor file in the youki crate. I checked with normal (non-wasm) image and a simple server in such cases is properly accessible. Also in your containerfile, you'll need to add a line EXPOSE 8000 for the container to actually expose the 8000 port (without it this won't work even if youki is correct). I tried adding that line, but its still not working, so need to see further in the wasmedge executor setup.

Edit : I checked with a simple replication of the executor outside the container, and that worked as expected, so error is somewhere in the container setup code, probably not executor

[Yashodhan]

Ok, I manager to recreate the exact same issue with same error with my smaller example. The key is , when run nomrally it works, but if I unshare the network namespace, it starts giving the error. So need to find what is the issue there next.

[Yashodhan]

@orsinium After looking a bit more, I have come to the conclusion that either

1. there is some issue with podman networking setup
2. the net package you have used has some issues when accessing across network namespaces

To confirm, I created a basic ubuntu container in podman (using runc/crun, not youki) mounted the wasm volume in it, and installed wasmedge. I also exposed the 8000 port when creating the cotnainer via podman run command.

If in the container I create a basic socat server by socat TCP-LISTEN:8000,crlf SYSTEM:"echo HTTP/1.0 200; echo Content-Type\\: text/plain; echo; cat" and try to connect it from outside with curl, it works (the curl shows error of request termination, but the socat prints the info of request, indicating that it was connected).

Now if I try to run the wasm with wasmedge, and do the same, curl shows it to be unreachable, and even the server do not log anything about getting a connection (I added the log in handling fn). However, if I exec into the container from another shell, and inside, I try to connect via curl, it works as expected.

So may I ask you to try compiling to wasm with some other library or language, and see if the above works? If it does AND it does not work with youki, then I'd be happy to investigate more ; but otherwise I'm inclined to close this issue as not something specifically related to youki.

Thanks for reporting none-the-less, it was sort of fun to do this debugging!

[Yashodhan]

Going ahead and closing this as not planned.

[Gram]

Thank you for looking into this! I'll try a different library and let you know how it goes. Sorry for the late reply, I just got back from vacation today.

[Yashodhan]

Thank you for looking into this! I'll try a different library and let you know how it goes.

👍

Sorry for the late reply, I just got back from vacation today.

No worries!