Enhancement::Need an option to NOT plumb an IP

[Navjot Singh]

This is an ENHANCEMENT request

Hi,

We are in the process of deploying our legacy application to cloud-native space. The application is exposed publicly by providing external IPs to the PODs. We cannot use L7 Ingress concept for exposure as the protocol is non-http (DIAMETER).

Moreover, the legacy peers, that will connect to our application on the external IP(s), cannot talk to more than one IP, neither they can do FQDN based discovery. Therefore, in order to achieve High Availability, the IP needs to be moved from one POD to another in case the former POD is temporarily unavailable/evicted/restarted etc. We have figured out a solution to handle this IP movement independent of the CNI.

The issue that we are facing is that since CNI specs mandates an IPAM, so when we configure whereabouts IPAM, there is no option to skip the IP plumbing. So basically, let's say we use multus + IPVLAN CNI to create an underlay interface, but we want that interface to be only L2 with no IP.

So as of now, since IP plumbing is mandatory, so we have to always plumb an extra static IP using whereabouts, and then plumb the actual floating-IP (which moves to another POD on failover), which will be used for the communication.

Our use-case is to support many-many IPs (around 44 in one deployment), so without this enhancement there is substantial IP wastage.

Please consider supporting this option.

[Doug Smith]

Is there a reason you want to do this specifically with Whereabouts?

Essentially, you can omit the IPAM and this should work -- I did a quick test with macvlan, and it works for not assigning an IP address.

Here's my example:

[centos@kube-singlehost-master ~]$ cat noip.crd.yml 
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: noip-conf
spec:
  config: '{
   "cniVersion":"0.3.1",
   "name":"outside-example",
   "type":"macvlan",
   "master":"eth0"
}'

[centos@kube-singlehost-master ~]$ cat noip.pod.yml 
apiVersion: v1
kind: Pod
metadata:
  name: noippod
  annotations:
    k8s.v1.cni.cncf.io/networks: noip-conf
spec:
  containers:
  - name: noippod
    command: ["/bin/bash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: centos/tools

[centos@kube-singlehost-master ~]$ kubectl create -f noip.crd.yml 
[centos@kube-singlehost-master ~]$ kubectl create -f noip.pod.yml 


[centos@kube-singlehost-master ~]$ kubectl exec -it noippod -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if27: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default 
    link/ether 2e:e5:a4:5d:ac:46 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.244.0.126/24 brd 10.244.0.255 scope global eth0
       valid_lft forever preferred_lft forever
4: net1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
    link/ether 8e:2c:a7:6c:cd:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0

[Navjot Singh]

Oh is it? Not sure what did I miss. I tried this thing with IPVLAN, but I remember clearly, it was expecting an IPAM.

[Doug Smith]

If it doesn't work with IPVLAN, I think it'd be a reasonable change to the CNI plugin itself, totally valid use case for L2 situation 👍

…

On Thu, Jan 14, 2021, 11:15 PM Navjot Singh ***@***.***> wrote: Oh is it? Not sure what did I miss. I tried this thing with IPVLAN, but I remember clearly, it was expecting an IPAM. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJX7PK5TCB6FESD53EVHY3SZ66PXANCNFSM4VXKXRYQ> .

[Navjot Singh]

Raised the above issue on IPvLAN. Thanks a lot for the test @dougbtv