Networking applications are of three types - Management applications, Control plane applications and data plane applications. Management and control plane applications are similar to Enterprise applications, but data plane applications different in following aspects:
- Multiple virtual network interfaces
- Multiple IP addresses
- SRIOV networking support
- Programmable virtual switch (for service function chaining, to tap the traffic for visibility etc..)
Kubernetes (Simply K8S) is the most popular container orchestrator. K8S is supported by GCE, AZURE and AWS and will be supported by Akraino Edge stack that enable edge clouds.
K8S has being enhanced to support VM workload types, this helps cloud providers that need to migrate legacy workloads to microservices architecture. Cloud providers may continue to support VM workload types for security reasons and hence there is need for VIM that support both VMs and containers. Since same K8S instance can orchestrate both VM and container workload types, same compute nodes can be leveraged for both VMs and containers. Telco and CSPs are seeing similar need to deploy networking applications as containers.
Since, both VMs and container workloads are used for networking applications, there would be need for
- Sharing the networks across VMs and containers.
- Sharing the volumes across VMs and containers.
Network Function Virtualization Requirements
NFV workloads can be,
- Management plane workloads
- Control plane work loads
- User plane (data plane workloads)
- User plane workloads normally have
- Multiple interfaces, Multiple subnets, Multiple virtual networks
- NFV workloads typically have its own management network.
- Some data plane workloads require SR-IOV NIC support for data interfaces and virtual NIC for other interfaces (for performance reasons)
- Need for multiple CNIs.
- NFV workloads require dynamic creation of virtual networks. Dynamic configuration of subnets.
This plugin addresses the below requirements, for networking workloads as well typical application workloads - Multi-interface support - Multi-IP address support - Dynamic creation of virtual networks - Co-existing with SRIOV and other CNIs. - Route management across virtual networks and external networks
OVN Background
OVN, the Open Virtual Network, is a system to support virtual network abstraction. OVN complements the existing capabilities of OVS to add native support for virtual network abstractions, such as virtual L2 and L3 overlays and security groups. Services such as DHCP are also desirable features. Just like OVS, OVN’s design goal is to have a production quality implementation that can operate at significant scale.
OVN4NFVK8s Plugin/ NFN-Operator development
ovn-kubernetes plugin is part of OVN project which provides OVN integration with Kubernetes but doesn't address the requirements as given above. To meet those requirements like multiple interfaces, IPs, dynamic creation of virtual networks, etc., OVN4NFVK8s plugin/ nfn-operator is created. It assumes that it will be used in conjuction with Multus_or other similar CNI which allows for the co-existance of multipleCNI plugins in runtime. This plugin assumes that the first interface in a Pod is provided by some other Plugin/CNI like Flannel or even OVN-Kubernetes. It is only responsible to add multiple interfacesbased on the Pod annotations. The code is based on ovn-kubernetes.
NFN-Operator has following functionalities: 1) It watches pods for annotations (see below) 2) It is a CRD Controller for dynamic networks, provider networks and
dynamic route creation.
- It is a gRPC server for nfn-agent running on all nodes in the cluster
nfn-operator uses operator-sdk and controller runtime for watching for pods and also other CR's. For creating dynamic logical networks Network CRD controller creates OVN logical switch as defined in CR. For provider network creation Provider Network CRD controller works with nfn-agent running on nodes to create provider network. nfn-operator communicates with nfn agent using gRPC. Currently only VLAN based provider networks are supported.
Note
This plugin is currently tested to work with Multus and Flannel providing the first network interface.
To meet the requirement of multiple interfaces and IP's per pod, a Pod annotation like below is required when working with Multus:
annotations:
k8s.v1.cni.cncf.io/networks: '[{ "name": "ovn-networkobj"}]'
k8s.plugin.opnfv.org/nfn-network: '{ "type": "ovn4nfv", "interface": [
{ "name": <name of OVN Logical Switch>, "interfaceRequest": "eth1" },
{ "name": <name of OVN Logical Switch>, "interfaceRequest": "eth2" }
]}'Based on these annotations watcher service in OVN4NFVK8s plugin/ nfn-operator assumes logical switch is already present. Dynamic IP addresses are assigned (static IP's also supported) and annotations are updated.
When the Pod is initialized on a node, OVN4NFVK8s CNI creates multiple interfaces and assigns IP addresses for the pod based on the annotations.
Multus Configuration Multus CRD definition for OVN:
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: ovn-networkobj
spec:
config: '{
"cniVersion": "0.3.1",
"name": "ovn4nfv-k8s-plugin",
"type": "ovn4nfvk8s-cni"
}'Please refer to Multus for details about how this configuration is used
CNI configuration file for Multus with Flannel:
{
"type": "multus",
"name": "multus-cni",
"cniVersion": "0.3.1",
"kubeconfig": "/etc/kubernetes/admin.conf",
"delegates": [
{
"type": "flannel",
"cniVersion": "0.3.1",
"masterplugin": true,
"delegate": {
"isDefaultGateway": false
}
}
]
}Refer Kubernetes documentation for the order in which CNI configurations are applied.
Build
For building the project:
cd ovn4nfv-k8s-plugin
make- This will output two files nfn-operator, nfn-agent and ovn4nfvk8s-cni which are the plugin/
- operator, gRPC client and CNI binaries respectively.
ovn4nfvk8s-cni requires some configuration at start up.
Example configuration file (default location/etc/openvswitch/ovn4nfv_k8s.conf)
[logging]
loglevel=5
logfile=/var/log/openvswitch/ovn4k8s.log
[cni]
conf-dir=/etc/cni/net.d
plugin=ovn4nfvk8s-cni
[kubernetes]
kubeconfig=/etc/kubernetes/admin.confCRD Controllers
nfn-operator includes controllers for 3 types of CRDs:
- Network CRD - To create logical networks.
- Provider Network CRD - To Create Provider networks. This works along with nfn-agent to create provider networks on nodes in cluster as needed.
- Chaining operator - To provision routes in Pods as per CR definition.
Network CR Example
apiVersion: k8s.plugin.opnfv.org/v1alpha1
kind: Network
metadata:
name: example-network
spec:
# Add fields here
cniType: ovn4nfv
ipv4Subnets:
- subnet: 172.16.44.0/24
name: subnet1
gateway: 172.16.44.1/24
excludeIps: 172.16.44.2 172.16.44.5..172.16.44.10Provider Network CR Example
apiVersion: k8s.plugin.opnfv.org/v1alpha1
kind: ProviderNetwork
metadata:
name: pnetwork
spec:
cniType: ovn4nfv
ipv4Subnets:
- subnet: 172.16.33.0/24
name: subnet1
excludeIps: 172.16.33.2 172.16.33.5..172.16.33.10
providerNetType: VLAN
vlan:
vlanId: "100"
providerInterfaceName: eth1
logicalInterfaceName: eth1.100
vlanNodeSelector: specific
nodeLabelList:
- kubernetes.io/hostname=testnode1Chaining CR Example
TODO
Figure
+-----------------+
| |
| | Program OVN Switch
|ovn4nfvk8s Plugin| +------------------+
| +--------------------->| |
| | | OVN Switch |
| | | |
| | +------------------+
+----+----------+-+
^ |
| |
|On Event |Annotate Pod
| |
| v
+----+--------------+ +------------------+ +-----------+
| | | | | Pod |
| Kube API +--------> Kube Scheduler +---------->| |
| | | | +--------+--+
| | +--------+---------+ |
+-------------------+ | |
| |
| |Assign IP & MAC
+--------v-----------+ |
| | |
| ovn4nfvk8s-cni | |
| +------------------+
+--------------------+
References
Authors/Contributors
Addepalli, Srinivasa R <srinivasa.r.addepalli@intel.com> Sood, Ritu <ritu.sood@intel.com>