Layer 3 Overlay Networks don't work on Windows

Question

Layer 3 Overlay Networks don't work on Windows

shynome opened this issue 2 years ago · comments

I has installed the TAP-windows 9.24.2
and the log is like below, but I can not ping the linux peer device 192.168.4.1. I had a try when between two linux device it is working.
What did I do wrong

./weron.exe vpn ip --community b --password x --key k --ips 192.168.4.3/24 --static
{"level":"info","addr":"wss://weron.herokuapp.com/","time":"2022-05-20T22:10:33+08:00","message":"Connecting to signaler"}
{"level":"info","id":"[\"192.168.4.3/24\"]","time":"2022-05-20T22:10:44+08:00","message":"Connected to signaler"}
{"level":"info","id":"[\"192.168.4.1/24\"]","time":"2022-05-20T22:10:44+08:00","message":"Connected to peer"}

Thanks for your reply and sorry my bad english

Felicitas Pojtinger · Answer 1 · Sat Jun 18 2022 02:39:03 GMT+0800 (China Standard Time)

Hi! Sorry for taking so long to respond. Could you post the debugging output by increasing the verbosity level (--verbose=7)? :)

shynome · Answer 2 · Sat Jun 18 2022 11:00:46 GMT+0800 (China Standard Time)

the log is like below and I still can not ping linux device

{"level":"info","addr":"wss://weron.herokuapp.com/","time":"2022-06-18T10:53:27+08:00","message":"Connecting to signaler"}
{"level":"trace","time":"2022-06-18T10:53:27+08:00","message":"Opening adapter"}
{"level":"debug","address":"wss://weron.herokuapp.com/?community=b&password=x","time":"2022-06-18T10:53:28+08:00","message":"Connected to signaler"}
{"level":"debug","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:28+08:00","message":"Claimed ID"}
{"level":"debug","address":"wss://weron.herokuapp.com/?community=b&password=x","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:28+08:00","message":"Introduced to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":97,"time":"2022-06-18T10:53:28+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:33+08:00","message":"Sending ping to signaler"}
{"level":"debug","id":"[\"192.168.4.3/24\"]","time":"2022-06-18T10:53:33+08:00","message":"Connected to signaler"}
{"level":"info","id":"[\"192.168.4.3/24\"]","time":"2022-06-18T10:53:33+08:00","message":"Connected to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:38+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:43+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:48+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:53+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:53:58+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:03+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:08+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:13+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","len":69,"community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received message from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received introduction from signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","channelID":"weron/ip/id","time":"2022-06-18T10:54:16+08:00","message":"Created data channel"}
{"level":"trace","address":"1.1.1.1:443","len":"udp4 host 192.168.56.1:62564","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Created ICE candidate"}
{"level":"trace","address":"1.1.1.1:443","len":"udp4 host 192.168.4.3:62565","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Created ICE candidate"}
{"level":"trace","address":"1.1.1.1:443","community":"b","channelID":"weron/ip/primary","time":"2022-06-18T10:54:16+08:00","message":"Created data channel"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":239,"time":"2022-06-18T10:54:16+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","len":"udp4 host 192.168.1.9:54516","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Created ICE candidate"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","client":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Sent ICE candidate to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":755,"time":"2022-06-18T10:54:16+08:00","message":"Sending message to signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","client":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Sent offer to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":235,"time":"2022-06-18T10:54:16+08:00","message":"Sending message to signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","client":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Sent ICE candidate to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":235,"time":"2022-06-18T10:54:16+08:00","message":"Sending message to signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","client":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Sent ICE candidate to signaler"}
{"level":"trace","address":"1.1.1.1:443","len":"udp4 srflx 171.35.83.90:32448 related 0.0.0.0:54517","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Created ICE candidate"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","client":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Sent ICE candidate to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":275,"time":"2022-06-18T10:54:16+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","len":211,"community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received message from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received candidate from signaler"}
{"level":"trace","address":"1.1.1.1:443","len":243,"community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received message from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received candidate from signaler"}
{"level":"trace","address":"1.1.1.1:443","len":247,"community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received message from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received candidate from signaler"}
{"level":"trace","address":"1.1.1.1:443","len":728,"community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received message from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:16+08:00","message":"Received answer from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Added answer from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Added ICE candidate from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Added ICE candidate from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:16+08:00","message":"Added ICE candidate from signaler"}
{"level":"trace","address":"1.1.1.1:443","len":247,"community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:17+08:00","message":"Received message from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:17+08:00","message":"Received candidate from signaler"}
{"level":"debug","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:17+08:00","message":"Added ICE candidate from signaler"}
{"level":"debug","label":"weron/ip/id","peer":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:17+08:00","message":"Connected to channel"}
{"level":"debug","label":"weron/ip/primary","peer":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:17+08:00","message":"Connected to channel"}
{"level":"debug","channelID":"weron/ip/id","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","candidates":0,"timestamp":1655520807060915600,"time":"2022-06-18T10:54:17+08:00","message":"Sending greeting"}
{"level":"debug","channelID":"weron/ip/id","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","id":"[\"192.168.4.3/24\"]","time":"2022-06-18T10:54:17+08:00","message":"Sending claimed"}
{"level":"debug","channelID":"weron/ip/id","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","time":"2022-06-18T10:54:17+08:00","message":"Received greeting"}
{"level":"debug","channelID":"weron/ip/id","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","id":"[\"192.168.4.3/24\"]","time":"2022-06-18T10:54:17+08:00","message":"Sending kick"}
{"level":"debug","channelID":"weron/ip/id","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","id":"","time":"2022-06-18T10:54:17+08:00","message":"Received kick"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:18+08:00","message":"Sending ping to signaler"}
{"level":"debug","channelID":"weron/ip/id","peerID":"01495a65-4ff5-4ad6-ae77-f76cfd1022c8","id":"[\"192.168.4.1/24\"]","time":"2022-06-18T10:54:20+08:00","message":"Received kick"}
{"level":"debug","channelID":"weron/ip/id","peerID":"[\"192.168.4.1/24\"]","id":"[\"192.168.4.1/24\"]","time":"2022-06-18T10:54:20+08:00","message":"Connected to peer"}
{"level":"debug","channelID":"weron/ip/primary","peerID":"[\"192.168.4.1/24\"]","time":"2022-06-18T10:54:20+08:00","message":"Connected to peer"}
{"level":"info","id":"[\"192.168.4.1/24\"]","time":"2022-06-18T10:54:20+08:00","message":"Connected to peer"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:23+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:28+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:33+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:38+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:43+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:48+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:53+08:00","message":"Sending ping to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","time":"2022-06-18T10:54:58+08:00","message":"Sending ping to signaler"}
{"level":"debug","time":"2022-06-18T10:54:59+08:00","message":"Gracefully shutting down"}
{"level":"trace","time":"2022-06-18T10:54:59+08:00","message":"Closing adapter"}
{"level":"trace","time":"2022-06-18T10:54:59+08:00","message":"Closing adapter"}
{"level":"trace","time":"2022-06-18T10:54:59+08:00","message":"Closing adapter"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","address":"1.1.1.1:443","community":"b","id":"65cfa023-d229-44d2-8c61-1fe20a4cdab2","len":28,"time":"2022-06-18T10:54:59+08:00","message":"Sending message to signaler"}
{"level":"trace","error":"context canceled","time":"2022-06-18T10:54:59+08:00","message":"Context cancelled"}

xg · Answer 3 · Wed Jun 22 2022 23:55:09 GMT+0800 (China Standard Time)

I'm exactly the same case. Peers are connected judged by logging; ping fails with timeout.

Felicitas Pojtinger · Answer 4 · Mon Nov 28 2022 02:07:39 GMT+0800 (China Standard Time)

Hmm, I had multiple reports of this issue now. I can't reproduce it locally. Have you tried pinging over IPv6?

Evans Jahja · Answer 5 · Thu Dec 22 2022 00:13:19 GMT+0800 (China Standard Time)

Hi, @pojntfx can reproduce issue with Windows <-> Windows
ping from weron works, but ping from command line doesn't work. Tried ipv4 and ipv6.

There's not much info even on verbose 7.

However, if I use vpn ethernet and manually assign the IP, it works neatly. I can ping and I can connect via TCP.

Therefore, I suspect the issue is only when using vpn ip

Evans Jahja · Answer 6 · Thu Dec 22 2022 01:26:51 GMT+0800 (China Standard Time)

@pojntfx
I tried reading the source code, and I think we might be missing a required parameter when calling water library

The PlatformSpecificParams has a property Network which is REQUIRED in Windows, but not in Linux.

https://github.com/songgao/water/blob/master/params_windows.go#L16-L26

Meanwhile, in wrtcip/netns_windows.go Network is not passed.

https://github.com/pojntfx/weron/blob/main/pkg/wrtcip/netns_windows.go#L11-L13

If you could try fixing that and build for windows, I'd be more than happy to try on my setup.

Felicitas Pojtinger · Answer 7 · Sun Apr 02 2023 07:30:30 GMT+0800 (China Standard Time)

@EvansJahja That's an interesting observation! We can't "just use" the IP here since its possible to assign multiple IPs in vpn ip mode, and this API only supports one - but it looks like we could also just a random range here and then set the proper ones with exec as we do now later, right?

Felicitas Pojtinger · Answer 8 · Sun Apr 02 2023 07:31:44 GMT+0800 (China Standard Time)

Pinned the issue in the meantime.

Evans Jahja · Answer 9 · Fri Aug 25 2023 16:42:55 GMT+0800 (China Standard Time)

@pojntfx I think you're right. In any case, the purpose of the Network is sort of "hacky", it's only there because Windows uses OpenVPN's wintap adapter that needs to know the subnet and network address in order to be able to respond to ARP. But putting the first IP address would probably be better (for windows users) rather than not having this functionality...

Felicitas Pojtinger · Answer 10 · Sun Aug 27 2023 00:26:03 GMT+0800 (China Standard Time)

Yeah, that's a good point. I was a bit tight on time recently because of academia, but I'll take a look at this ASAP. I luckily still have access to a Windows machine rn, might as well use it ^^

Felicitas Pojtinger · Answer 11 · Mon Aug 28 2023 03:19:31 GMT+0800 (China Standard Time)

This required quite an elaborate fix, but I just tested it on a setup where one Linux (Fedora 38) client and one Windows 11 client were able to communicate with each other with the L3 VPN after this commit: e1ed0c4. Basically, this does what @EvansJahja pointed out, which required reimplementing the way that IP addresses are assigned; I used this to verify:

Fedora 38:

$ weron vpn ip --community mycommunity --password mypassword --key mykey --ips 192.0.2.1/24

Windows 11:

PS> weron.exe vpn ip --community mycommunity --password mypassword --key mykey --ips 192.0.2.1/24

Felicitas Pojtinger · Answer 12 · Mon Aug 28 2023 03:20:24 GMT+0800 (China Standard Time)

@EvansJahja Would you mind confirming that this fix works on your setup, too? You can download the fixed binary from here: https://github.com/pojntfx/weron/releases/tag/unstable

Felicitas Pojtinger · Answer 13 · Wed Sep 13 2023 09:36:22 GMT+0800 (China Standard Time)

This should be fixed now - weron v0.2.6 now has fully Windows support for both Layer 2 and 3 🥳

Feel free to reopen this issue if it reoccurs ^^