Networking Question
segator opened this issue · comments
First of all I want to tell you, Thank you for your Guide!!
let's ask.
you say you need to add and extra IP to the WG device.
ip route add 10.96.0.0/16 dev $VPN_INTERFACE src $VPN_IP
# on kube1:
ip route add 10.96.0.0/16 dev wg0 src 10.0.1.1
# on kube2:
ip route add 10.96.0.0/16 dev wg0 src 10.0.1.2
# on kube3:
ip route add 10.96.0.0/16 dev wg0 src 10.0.1.3Why is this needed and why 10.96.0.0? this is by default weaveNet range?
weaveNet as far as I know goes through UDP and crypt, this should create and extra overhead that doesnt needed. Maybe is better user flannel hostgw or calico?
I'm trying to build a kubernetes with 3 nodes on 3 diferents locations. so connection between nodes work through internet.
10.96.0.0/16 is indeed the default IP range used by WeaveNet.
Using WireGuard to create a VPN overlay is perfect for what you want to achieve and from there it's up to you which k8s networking solution you'd like to use.
I finally got it Working with wireguard + flannel host-gateway
I tested multiple CNI's
Finally choosed
flannel host-gateway far better performance than weave Net
@segator : could you explain how you tested and what you needed to configure to get flannel working (instead of Weave Net)?
I wrote a simple application to do the automatic mesh of wireguard
https://github.com/segator/wireguard-dynamic
Then when installing workers remember to use their wireguard IP, so you will force the system to use wireguard.
And then when deploying the flannel network change in the yml from vxlan to host-gw and apply the file.
This all depends of the kube deploy tool you are using, I'm using RKE right now so is quite easy, for kubeadm not sure I supose will be something similar than https://prefetch.net/blog/2018/02/20/getting-the-flannel-host-gw-working-with-kubernetes/
Hope it helps!