Giters

k8snetworkplumbingwg / whereabouts

A CNI IPAM plugin that assigns IP addresses cluster-wide

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Issue with host-device as main plugin and whereabouts as IPAM

nelsonpraveen opened this issue · comments

commented

Hi,
I face an issue when using host-device main plugin in Multus and whereabouts as IPAM.

The network-attachment-definition is given below.

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: hostdev-conf
spec:
  config: '{
      "cniVersion": "0.3.1",
      "name": "whereaboutsexample",
      "type": "host-device",
      "device": "ens1f1",
      "vlan": 1625,
      "if0name": "eth1",
      "ipam": {
        "type": "whereabouts",
        "datastore": "kubernetes",
        "range": "x.x.x.x/27",
        "range_start": "x.x.x.x",
        "range_end": "x.x.x.x",
        "gateway": "x.x.x.x",
        "log_file" : "/tmp/whereabouts.log",
        "log_level" : "debug"
      }
    }'

When I try to deploy the network and use in a POD, the net device in the worker node seems to get deactivated which seems strange.

The log from syslog is:

May 21 00:52:33 worker7 kernel: [40450.134695] IPv6: ADDRCONF(NETDEV_CHANGE): ens1f1: link becomes ready
May 21 00:52:33 worker7 kernel: [40450.135228] IPv6: ADDRCONF(NETDEV_CHANGE): ens1f1.1625: link becomes ready
May 21 00:52:33 worker7 kernel: [40450.135919] IPv6: ADDRCONF(NETDEV_CHANGE): ens1f1.1626: link becomes ready
May 21 00:52:35 worker7 charon: 08[KNL] fe80::92e2:baff:feb6:5fd1 appeared on ens1f1.1625
May 21 00:52:35 worker7 charon: 11[KNL] fe80::92e2:baff:feb6:5fd1 appeared on ens1f1
May 21 00:52:35 worker7 charon: 10[KNL] fe80::92e2:baff:feb6:5fd1 appeared on ens1f1.1626
**May 21 00:52:36 worker7 kernel: [40452.850409] ixgbe 0000:03:00.1: removed PHC on ens1f1
May 21 00:52:36 worker7 charon: 10[KNL] interface ens1f1 deactivated**
May 21 00:52:36 worker7 charon: 09[KNL] fe80::92e2:baff:feb6:5fd1 disappeared from ens1f1
May 21 00:52:36 worker7 charon: 07[KNL] interface ens1f1.1626 deactivated
May 21 00:52:36 worker7 charon: 13[KNL] fe80::92e2:baff:feb6:5fd1 disappeared from ens1f1.1626
May 21 00:52:36 worker7 charon: 11[KNL] interface ens1f1.1625 deactivated
May 21 00:52:36 worker7 charon: 05[KNL] fe80::92e2:baff:feb6:5fd1 disappeared from ens1f1.1625
May 21 00:52:36 worker7 charon: 06[KNL] interface ens1f1 deleted

The device which was UP and fine becomes 'DOWN' somehow - not sure if whereabouts causes this.

ip link | grep ens1f1
5: ens1f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state **DOWN** mode DEFAULT group default qlen 1000
    alias ens1f1
14: ens1f1.1625@ens1f1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noqueue state **DOWN** mode DEFAULT group default qlen 1000
    alias ens1f1.1625
15: ens1f1.1626@ens1f1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noqueue state **DOWN** mode DEFAULT group default qlen 1000

The whereabouts log shows this:

2021-05-21T01:48:39-04:00 [debug] Beginning IPAM for ContainerID: 98d2c3baf40b6b91f8f4717509d58e998caed82ab755eceab2d5aa4303cba339
2021-05-21T01:48:39-04:00 [debug] IPManagement -- mode: 0 / host:  / containerID: 98d2c3baf40b6b91f8f4717509d58e998caed82ab755eceab2d5aa4303cba339
2021-05-21T01:48:39-04:00 [error] IPAM kubernetes client initialization error: Get https://10.96.0.1:443/api?timeout=32s: dial tcp 10.96.0.1:443: connect: network is unreachable
2021-05-21T01:48:39-04:00 [error] Error at storage engine: IPAM kubernetes client initialization error: Get https://10.96.0.1:443/api?timeout=32s: dial tcp 10.96.0.1:443: connect: network is unreachable
2021-05-21T01:48:39-04:00 [debug] Used defaults from parsed flat file config @ /etc/cni/net.d/whereabouts.d/whereabouts.conf
2021-05-21T01:48:39-04:00 [debug] DEL - IPAM configuration successfully read: {Name:whereaboutsexample Type:whereabouts Routes:[] Datastore:kubernetes Addresses:[] OmitRanges:[] DNS:{Nameservers:[] Domain: Search:[] Options:[]} Range:14.1.8.0/27 RangeStart:x.x.x.x RangeEnd:x.x.x.x GatewayStr:x.x.x.x EtcdHost: EtcdUsername: EtcdPassword:********* EtcdKeyFile: EtcdCertFile: EtcdCACertFile: LogFile:/tmp/whereabouts.log LogLevel:debug Gateway:14.1.8.1 Kubernetes:{KubeConfigPath:/etc/cni/net.d/whereabouts.d/whereabouts.kubeconfig K8sAPIRoot:} ConfigurationPath:}
2021-05-21T01:48:39-04:00 [debug] ContainerID: 98d2c3baf40b6b91f8f4717509d58e998caed82ab755eceab2d5aa4303cba339
2021-05-21T01:48:39-04:00 [debug] IPManagement -- mode: 1 / host:  / containerID: 98d2c3baf40b6b91f8f4717509d58e998caed82ab755eceab2d5aa4303cba339
2021-05-21T01:48:39-04:00 [error] IPAM kubernetes client initialization error: Get https://10.96.0.1:443/api?timeout=32s: dial tcp 10.96.0.1:443: connect: network is unreachable
2021-05-21T01:48:39-04:00 [verbose] WARNING: Problem deallocating IP: IPAM kubernetes client initialization error: Get https://10.96.0.1:443/api?timeout=32s: dial tcp 10.96.0.1:443: connect: network is unreachable

Few points worth mentioning here:

  1. I tried the same network device and worker node with host-device plugin and host-local IPAM. It works fine.
  2. For same network with sriov and ipvlan plugins, whereabouts works fine

The issue seems to be only with combination of host-device plugin and whereabouts IPAM.

Is there any additional configuration I am missing or some error in my network definition file ?

commented

The same combination works in a VM setup. This issue occurs only in baremetal setup which must be because of host-device plugin (sriov is used in BM setup). Closing the issue