Connectivity issues between 2 specific nodes after ~5 minutes

[Luka]

What is the issue?

I’m having many issues at the same time since 2 days ago, probably related to 1.66 auto-update of my exit node.

After more than a year of flawless network configuration, I struggled hard to understand why my IP forwarding iptables rules on the exit node were not working anymore, before learning about the new --stateful-filtering=false.

However, even with the flag enabled, I’m facing a weird behavior where my 2 most important nodes (the exit node with public IPs, and the client node running a few servers such as a emailing) can’t ping each other after a few minutes (~300s to ~500s). To fix it, I have to restart tailscaled on the client node.

Current workaround is the following cronjob: */2 * * * * systemctl restart tailscaled.

Steps to reproduce

Start tailscale on both nodes with the following configuration:

Client:

tailscale up --exit-node-allow-lan-access --exit-node=[exit node IP] --snat-subnet-routes=false

Exit node:

tailscale up --advertise-exit-node --accept-routes --snat-subnet-routes=true --stateful-filtering=false

Are there any recent changes that introduced the issue?

Recent auto-update on exit-node to 1.66 and more than a dozen hours of investigation, debug, and tests. I rolled back to original configuration once I learned about --stateful-filtering=false.

OS

Linux

OS version

NixOS 23.11 (client)/Debian 10.13 (exit node)

Tailscale version

1.64.2 (client)/1.66.1 (exit node)

Other software

iptables on client (default NixOS behavior for port filtering), and on exit node (for IP forwarding).

Exit node iptables looks like this:

-A PREROUTING -d [PubIP]/32 -p tcp -j DNAT --to-destination [ClientIP]
-A POSTROUTING -s [ClientIP]/32 -j SNAT --to-source [PubIP]
-A OUTPUT -d [PubIP]/32 -p tcp -j DNAT --to-destination [ClientIP]
-A POSTROUTING -j MASQUERADE

Bug report

BUG-d2d04016a75c0316f48b060d8fa760072d3ddd01fae3090532d240003937884f-20240511130410Z-306634be34dfbfc6