Connection originates from a wrong IP when using VPN

Question

Connection originates from a wrong IP when using VPN

rkfg opened this issue 9 years ago · comments

The code is as simple as this:

        KeyPairGenerator keygen = KeyPairGenerator.getInstance("DSA");
        KeyPair keyPair = keygen.generateKeyPair();
        Peer basePeer = new PeerBuilder(Utils.makeSHAHash(keyPair.getPublic().getEncoded())).keyPair(keyPair).ports(7700).start();
        PeerDHT dhtPeer = new PeerBuilderDHT(basePeer).start();
        InetAddress inetAddr = InetAddress.getByName(args[0]);
        FutureDiscover discover = dhtPeer.peer().discover().inetAddress(inetAddr).start();
        discover.awaitUninterruptibly();

I'm getting a timeout because connection initiates from a wrong interface as seen in Wireshark. Here's what I have:

br0 (a bridge interface, contains only eth0, using it instead of just eth0 for docker/virtual machines/etc.): 172.25.10.35/24
tun0 (VPN provider): 10.6.0.29/16

Internet is available on both interfaces but routing is set up to route all traffic through tun0 (except the VPN server I connect to). It's a bit weird:
0.0.0.0/1 via 10.6.0.1 dev tun0
default via 172.25.10.36 dev br0
10.6.0.0/16 dev tun0 proto kernel scope link src 10.6.0.29
172.25.10.0/24 dev br0 proto kernel scope link src 172.25.10.35
128.0.0.0/1 via 10.6.0.1 dev tun0
178.xxx.xxx.xxx via 172.25.10.36 dev br0

172.25.10.36 is my router, 178.xxx.xxx.xxx is the VPN server. This solution allows to replace the default route without actually removing it by creating two /1 subnets that together fully cover the IPv4 address space and have priority over /0 default route. That's not my invention, it's how the VPN provider set it up. Enough with these technical details, sufficient to say I don't have any issues with it and all network software just works.

So, TomP2P tries to connect to some Internet IP via TCP from br0 interface but sends the SYN packet originating from 10.6.0.29! The source IP is ~~valid but the interface is not, it should be tun0 instead~~ invalid while the source interface is fine (explained in further comments). Of course, my router is still accessible and probably TomP2P discovers it via UPnP but all the traffic goes through VPN. I tried .bindings(new Bindings().addInterface("tun0")) but still there's no traffic on tun0 (and it's also fine, tun0 shouldn't be involved, explained further).

Thomas Bocek · Answer 1 · Fri Oct 16 2015 00:17:38 GMT+0800 (China Standard Time)

A timeout ("Timeout in Discover: ...") in discovery() means that your peer is behind a NAT or firewall. Discovery contacts a peer that is well reachable in the Internet and tells this peer to contact my peer behind the NAT. The well reachable peer will most likely see the IP 178.* and try to contact this peer on port 7700. Since this is your VPN server, the packets will be dropped and thus you see a timeout.

So seeing a timeout is normal when your peer is not reachable.

rkfg · Answer 2 · Fri Oct 16 2015 00:24:23 GMT+0800 (China Standard Time)

Could be it but why can't I connect to that peer? I'm doing bootstrap later and still have no traffic on tun0:

        FutureBootstrap fb = dhtPeer.peer().bootstrap().inetAddress(inetAddr).ports(7700).start();
        fb.awaitUninterruptibly();

Also, forgot to provide the log:

15 Oct 2015 19:02:24,514 [] DEBUG Reservation - channel from /10.6.0.29
15 Oct 2015 19:02:24,527 [] DEBUG ChannelCreator - Create TCP, use from address: /10.6.0.29
15 Oct 2015 19:02:24,532 [] DEBUG Sender - about to connect to paddr[0xrkfg.me/37.48.67.11,7700,] with channel [id: 0x7402f7bc, /10.6.0.29:40854], ff=false, msg=msgid=-445226265,t=REQUEST_2,c=PING,tcp,s=paddr[0xd112fcf5f0f219106ab4655e512840c5cd3f73b8/10.6.0.29,7700,],r=paddr[0xrkfg.me/37.48.67.11,7700,]
15 Oct 2015 19:02:27,537 []  WARN Sender - Channel creation failed
io.netty.channel.ConnectTimeoutException: connection timed out: rkfg.me/37.48.67.11:7700
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:220)
    at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
    at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)
15 Oct 2015 19:02:27,539 []  WARN Sender - Faild message msgid=-445226265,t=REQUEST_2,c=PING,tcp,s=paddr[0xd112fcf5f0f219106ab4655e512840c5cd3f73b8/10.6.0.29,7700,],r=paddr[0xrkfg.me/37.48.67.11,7700,]
15 Oct 2015 19:02:27,540 [] DEBUG Sender - peer failed: msgid=-445226265,t=REQUEST_2,c=PING,tcp,s=paddr[0xd112fcf5f0f219106ab4655e512840c5cd3f73b8/10.6.0.29,7700,],r=paddr[0xrkfg.me/37.48.67.11,7700,], future response state:,type:FAILED,msg:0,reason:Channel creation failed [id: 0x7402f7bc, /10.6.0.29:40854]/io.netty.channel.ConnectTimeoutException: connection timed out: rkfg.me/37.48.67.11:7700
15 Oct 2015 19:02:27,540 [] DEBUG PeerMap - Peer paddr[0xrkfg.me/37.48.67.11,7700,] is offline with reason PeerException (PEER_ERROR): Future (compl/canc):true/, FAILED, Channel creation failed [id: 0x7402f7bc, /10.6.0.29:40854]/io.netty.channel.ConnectTimeoutException: connection timed out: rkfg.me/37.48.67.11:7700.
15 Oct 2015 19:02:27,540 [] ERROR Main - Discovery failed: Future (compl/canc):true/, FAILED, FutureDiscover (1): We need at least the TCP connection <-> Future (compl/canc):true/, FAILED, Channel creation failed [id: 0x7402f7bc, /10.6.0.29:40854]/io.netty.channel.ConnectTimeoutException: connection timed out: rkfg.me/37.48.67.11:7700
15 Oct 2015 19:02:27,543 [] DEBUG Reservation - channel from /10.6.0.29
15 Oct 2015 19:02:27,544 [] DEBUG ChannelCreator - Create TCP, use from address: /10.6.0.29
15 Oct 2015 19:02:27,544 [] DEBUG Sender - about to connect to paddr[0xrkfg.me/37.48.67.11,7700,] with channel [id: 0x363153bc], ff=false, msg=msgid=1535857738,t=REQUEST_1,c=PING,tcp,s=paddr[0xd112fcf5f0f219106ab4655e512840c5cd3f73b8/10.6.0.29,7700,],r=paddr[0xrkfg.me/37.48.67.11,7700,]
15 Oct 2015 19:02:30,545 []  WARN Sender - Channel creation failed
io.netty.channel.ConnectTimeoutException: connection timed out: rkfg.me/37.48.67.11:7700
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:220)
    at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
    at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)
15 Oct 2015 19:02:30,545 []  WARN Sender - Faild message msgid=1535857738,t=REQUEST_1,c=PING,tcp,s=paddr[0xd112fcf5f0f219106ab4655e512840c5cd3f73b8/10.6.0.29,7700,],r=paddr[0xrkfg.me/37.48.67.11,7700,]
15 Oct 2015 19:02:30,545 [] DEBUG Sender - peer failed: msgid=1535857738,t=REQUEST_1,c=PING,tcp,s=paddr[0xd112fcf5f0f219106ab4655e512840c5cd3f73b8/10.6.0.29,7700,],r=paddr[0xrkfg.me/37.48.67.11,7700,], future response state:,type:FAILED,msg:0,reason:Channel creation failed [id: 0x363153bc]/io.netty.channel.ConnectTimeoutException: connection timed out: rkfg.me/37.48.67.11:7700
15 Oct 2015 19:02:30,545 [] DEBUG PeerMap - Peer paddr[0xrkfg.me/37.48.67.11,7700,] is offline with reason PeerException (PEER_ERROR): Future (compl/canc):true/, FAILED, Channel creation failed [id: 0x363153bc]/io.netty.channel.ConnectTimeoutException: connection timed out: rkfg.me/37.48.67.11:7700.

And I can connect to that ip/port via telnet, no problem.

Thomas Bocek · Answer 3 · Fri Oct 16 2015 00:45:22 GMT+0800 (China Standard Time)

Interesting, its not a regular discovery timeout issue, but it seems that you cannot establish a TCP connection to rkfg.me.

In the latest commit, I exposed now fromAddress in ChannelClientConfiguration. Can you try to set it to 0.0.0.0 and see what happens?

rkfg · Answer 4 · Fri Oct 16 2015 03:43:22 GMT+0800 (China Standard Time)

Thanks, this way it works fine. Here's what I added:

Peer basePeer = new PeerBuilder(Utils.makeSHAHash(keyPair.getPublic().getEncoded()))
                .keyPair(keyPair)
                .ports(7700)
                .channelClientConfiguration(
PeerBuilder.createDefaultChannelClientConfiguration().fromAddress(InetAddress.getByName("0.0.0.0"))).start();

rkfg · Answer 5 · Fri Oct 16 2015 03:59:24 GMT+0800 (China Standard Time)

However, it's very strange. TomP2P seems to connect from my real address, not via VPN. I've captured traffic on the router and it's clearly the case. I didn't know it's possible at all considering the routing...

My bad, forgot that I set the direct route to my own server. Maybe that's what caused the confusion, the default route pointed to the tun0 interface while the target peer was accessible directly via br0 because of that one route?

So far, looks good to me though that 0.0.0.0 fromAddress should definitely be set by default.

Thomas Bocek · Answer 6 · Fri Oct 16 2015 04:26:32 GMT+0800 (China Standard Time)

Thanks for testing.

Hmm, I had issues in my NAT setup with 0.0.0.0, I need to investigate this. Most likely I need to open two connections and see what is coming back.

With 0.0.0.0, which interface was used? Is it going through br0? The thing in Java is that you cannot set a sending interface, just set an inetaddress and it will figure out the interface. I wonder why this is not the case with 10.6.0.29. This should be clearly going out through tun0.

rkfg · Answer 7 · Fri Oct 16 2015 05:46:13 GMT+0800 (China Standard Time)

It's going through the right interface, br0 in this case with the direct route (for my server) added. After I remove it, it also works fine switching to tun0. Everything is as it should be, double checked with Wireshark.

Actually, the situation in the first post should be described a bit more correctly. The problem was not the interface but the source address. As I said, I forgot about that "exclusion" route that allows me to bypass VPN for my server so any traffic to it goes from br0 and it was actually right. The wrong thing was the IP selected as the source, 10.6.0.29, while it should have been 172.25.10.35.

And another one little thing:
16 Oct 2015 00:46:10,241 [] INFO PeerCreator - Status of external address search: Discover status: ++tun0( /10.6.0.29), ++tinc0( /10.10.0.101), ++br0( /172.25.10.35), ++lo( /127.0.0.1).

Could it be that TomP2P takes the first discovered interface as the source "by default"?

rkfg · Answer 8 · Fri Oct 16 2015 07:40:18 GMT+0800 (China Standard Time)

Fixed the issue title and the first comment.

Thomas Bocek · Answer 9 · Thu Oct 22 2015 17:49:46 GMT+0800 (China Standard Time)

Yes, TomP2P takes the first non-local interface. To make TomP2P robust for "exotic" network configuration, I should test various source addresses and see what is coming through.