get the opposite result

[Solaris]

Hi @jackkleeman,
After I deploymented this operator, I get the opposite result.I can't access baidu.com, but others site can. I expect that i can only access baidu.com. How can i debug this, thanks.

kubectl -n egress-operator-system get po
NAME READY STATUS RESTARTS AGE
baidu-6499d5495b-g5t87 1/1 Running 0 54m
egress-operator-controller-manager-6f99db96f4-v45k5 2/2 Running 0 8d

kubectl -n egress-operator-system logs baidu-6499d5495b-g5t87
...

[2020-02-01T08:48:59.603Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-"
[2020-02-01T08:49:09.602Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-"
[2020-02-01T08:49:19.602Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-"
[2020-02-01T08:49:29.602Z] "GET /ready HTTP/1.1" 200 - 0 5 0 - "10.209.156.230" "kube-probe/1.13" "-" "10.32.120.102:11000" "-"
...

kubectl -n egress-operator-system logs egress-operator-controller-manager-6f99db96f4-v45k5 -c manager
...
2020-02-01T07:39:38.361Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"}
2020-02-01T07:39:38.460Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"}
2020-02-01T07:39:53.271Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"}
2020-02-01T07:40:59.660Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "externalservice", "request": "/baidu"}
...

kubectl -n egress-operator-system get networkpolicy
NAME POD-SELECTOR AGE
baidu egress.monzo.com/gateway=baidu 67m
egress-operator-public-egress app=egress-gateway 8d

the CoreDNS config:
.:53 {
errors
health
egressoperator egress-operator-system cluster.local
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
reload
loadbalance
}

the ExternalService yaml:

apiVersion: egress.monzo.com/v1
kind: ExternalService
metadata:
  name: baidu
spec:
  dnsName: baidu.com
  hijackDns: true
  ports:
  - port: 80
  - port: 443
    protocol: TCP
  minReplicas: 1
  maxReplicas: 3
  targetCPUUtilizationPercentage: 30
  resources:
    requests:
      cpu: 100m
      memory: 100Mi
    limits:
      cpu: 500m
      memory: 200Mi

the pod yaml:

kind: Pod
apiVersion: v1
metadata:
  name: test-pod
  labels:
    egress.monzo.com/gateway: baidu
spec:
  containers:
  - name: test-pod
    image: alpine
    command:
    - "/bin/sleep"
    args:
    - "36000"
  restartPolicy: "Never"

[Jack Kleeman]

This operator won't block other sites for you, you'll need a default deny egress network policy which allows internal traffic but not external traffic, in namespaces you want to control. If you do that, then only traffic through gateways (which looks like an internal IP) will be allowed

I've updated the readme to give an example default deny egress policy

[Jack Kleeman]

I see the problem - you have the wrong label on your pod - should be egress.monzo.com/allowed-gateway: baidu

[Solaris]

Hi @jackkleeman
I have followed as you said, but now i also can't access the external server like baidu.com

[Solaris]

I find that in baidu-6499d5495b-g5t87, i can't access the dns, and i can't access any site, include the host ip.

[Solaris]

I have use cilium to solve it

[Jack Kleeman]

I suspect that your network policy was too strict and it wasn't allowing internal traffic.