Dual-stack services fail
bootc opened this issue · comments
If I create a dual-stack LoadBalancer service (say, using Nordix/assign-lb-ip), network-node-manager is confused by this and tries to add IPv4 iptables rules for the IPv6 external IP:
2021-03-18T23:27:07.341Z INFO controllers.Service.reconcile create iptables rules {"service": "bootc-system/ingress-nginx-controller", "externalIP": "192.0.2.52", "clusterIP": "10.43.150.86"}
2021-03-18T23:27:07.441Z INFO controllers.Service.reconcile create iptables rules {"service": "bootc-system/ingress-nginx-controller", "externalIP": "2001:db8::5", "clusterIP": "10.43.150.86"}
2021-03-18T23:27:07.445Z ERROR controllers.Service.reconcile iptables v1.8.3 (legacy): host/network `2001:db8::5' not found
Try `iptables -h' or 'iptables --help' for more information.
{"service": "bootc-system/ingress-nginx-controller", "error": "exit status 2"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
github.com/kakao/network-node-manager/pkg/rules.CreateRulesExternalCluster
/workspace/pkg/rules/rule_external_cluster.go:314
github.com/kakao/network-node-manager/controllers.(*ServiceReconciler).Reconcile
/workspace/controllers/service_controller.go:208
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:256
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88
2021-03-18T23:27:07.445Z ERROR controller-runtime.controller Reconciler error {"controller": "service", "request": "bootc-system/ingress-nginx-controller", "error": "exit status 2"}
github.com/go-logr/zapr.(*zapLogger).Error
/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:258
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
/go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88
The service that triggers the above looks like this (redacted for clarity):
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx-controller
namespace: bootc-system
spec:
clusterIP: 10.43.150.86
clusterIPs:
- 10.43.150.86
- 2001:db8:ff::edd5
externalIPs:
- 192.0.2.52
- 2001:db8::5
externalTrafficPolicy: Local
ipFamilies:
- IPv4
- IPv6
ipFamilyPolicy: RequireDualStack
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 192.0.2.52
- ip: 2001:db8::5It seems like the controller needs to look at the clusterIPs field rather than the singular clusterIP field and match the address family for the externalIP it is looking at.
@bootc Yes. Now network-node-manager doesn't support dual-stack in a service. I will work for this problem. Thanks :)
I actually have a work-in-progress branch for this, if you can hang on a few hours.
I have a Docker image with the above available at bootc/network-node-manager:issue-10.