Having trouble using Telepresence with Tailscale

[Francesco Virga]

Describe the bug
Our team has been trying to integrate Telepresence into our workflow, but we have not been able to get it to play nicely with Tailscale. Telepresence gives us a success message when we run telepresence connect, but we can't actually access any k8s services. When we try it with the VPN disconnected, it works fine.

We've followed instructions from here: https://www.getambassador.io/docs/telepresence/latest/reference/vpn. Here's what our setup looks like:

• GKE cluster using 10.96.0.0/14 pod IP range and 10.100.0.0/20 services IP range
• Tailscale running in a GCP VM. It assigns IP addresses to clients somewhere in the 100.X.X.X range
• Ran telepresence helm upgrade --set client.routing.allowConflictingSubnets="{10.96.0.0/14,10.100.0.0/20}"
• Added 10.96.0.0/14 and 10.100.0.0/20 as subnet routes in the Tailscale settings for the exit node

What are we missing? Any insight would be super useful.

Telepresence logs:
telepresence_logs.zip

To Reproduce
Steps to reproduce the behavior:

1. Connect to VPN
2. Run telepresence connect
3. Can't access cluster resources

Expected behavior
Can access cluster resources

Versions (please complete the following information):

• Output of telepresence version:
  Client : v2.18.1
  Root Daemon : v2.18.1
  User Daemon : v2.18.1
  Traffic Manager: v2.18.1

• Operating system of workstation running telepresence commands
  MacOS Ventura 13.4

• Kubernetes environment and Version [e.g. Minikube, bare metal, Google Kubernetes Engine]
  GKE, version 1.27.8-gke.1067004

VPN-related bugs:
If you're reporting an issue around telepresence connectivity when using a VPN,
and are running Telepresence 2.4.8 or above, please also attach the output
of telepresence test-vpn, and the following information:

Tried running telepresence test-vpn but got an error that its deprecated.

• Which VPN client are you using?
  Tailscale
• Which VPN server are you using?
  Tailscale
• How is your VPN pushing DNS configuration? It may be useful to add the contents of /etc/resolv.conf
  Uses Tailscale MagicDNS, running on IP 100.100.100.100

Additional context
Add any other context about the problem here.

[Thomas Hallgren]

There's no conflict between subnet 100. (Tailscape) and subnets starting with 10. unless one of them has a very small mask (just 2 bits, which is unusual).

I don't understand why the allowConflictingSubnets setting is at all needed, or why you would be adding the GKE cluster subnets to Tailscape.

Can you please elaborate on why you're trying this? In what way is Tailscape interfering with the cluster's subnet?

[Francesco Virga]

I tried adding it because telepresence would stop resolving to cluster DNS every time I connected to my vpn. I agree it doesn’t really make sense to add since the IP ranges don’t overlap but I didn’t know what else could be conflicting so I did just in case.

I’ve gone through the docs and tailscale settings but haven’t found anything else that could be causing issues. I’m pretty new to telepresence so wondering if I’m missing something, but again I’ve combed through the docs and haven’t found anything obvious.

[Thomas Hallgren]

I believe that what you see here is a DNS issue. Tailscale installs a somewhat intrusive DNS resolver and it will override Telepresence's DNS. I believe that you can configure Tailscale to not do that.

[Francesco Virga]

@thallgren Do you have any idea how I'd do this? I dont see any obvious options in the Tailscale DNS settings

[cindymullins-dw]

Wondering if its Tailscale DNS outright, or something to do with MagicDNS. Have you tried disabling MagicDNS and does Telepresence work any differently in that case?

[Francesco Virga]

Ive been experimenting today and it seems to have fixed the problem, thanks! I will re-open this if the problem re-occurs