monzo / egress-operator

A Kubernetes operator to produce egress gateway Envoy pods and control access to them with network policies

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

curl: (35) Unknown SSL protocol error in connection to github.com:443

srinicrick65 opened this issue · comments

hi ,
When i am installing calico as CNI plugin for kubernetes k3s distribution and try the curl on the domain which has to go through egress gateway i am getting SSL issue . I am not sure what is the issue but i can see that the call is redirecting to the egress service. But when i am not having calico cni it works fine. Can someone help with this .

Requirement

curl 7.52.1
OpenSSL 1.1.0l 10 Sep 2019 (Library: OpenSSL 1.1.0j 20 Nov 2018)
K3S_VERSION=v1.21.1+k3s1
Calico as CNI

root@nginx:/# curl https://github.com -v

  • Rebuilt URL to: https://github.com/
  • Trying 10.43.243.74...
  • TCP_NODELAY set
  • Connected to github.com (10.43.243.74) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@strength
  • successfully set certificate verify locations:
  • CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: /etc/ssl/certs
  • TLSv1.2 (OUT), TLS header, Certificate Status (22):
  • TLSv1.2 (OUT), TLS handshake, Client hello (1):
  • Unknown SSL protocol error in connection to github.com:443
  • Curl_http_done: called premature == 0
  • Closing connection 0
    curl: (35) Unknown SSL protocol error in connection to github.com:443
root@srini-Virtual-Machine:~/Documents/acl_manifest# kubectl get pods -A -o wide
NAMESPACE                NAME                                                  READY   STATUS    RESTARTS   AGE     IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system              calico-node-tdlx4                                     1/1     Running   0          6h13m   192.168.0.118   srini-virtual-machine   <none>           <none>
kube-system              local-path-provisioner-5ff76fc89d-vjm9f               1/1     Running   0          6h13m   172.18.106.1    srini-virtual-machine   <none>           <none>
kube-system              metrics-server-86cbb8457f-gs64v                       1/1     Running   0          6h13m   172.18.106.2    srini-virtual-machine   <none>           <none>
kube-system              calico-kube-controllers-78d6f96c7b-6t6br              1/1     Running   0          6h13m   172.18.106.3    srini-virtual-machine   <none>           <none>
kube-system              coredns-b679fff67-4l4jj                               1/1     Running   0          6h6m    172.18.106.5    srini-virtual-machine   <none>           <none>
egress-operator-system   egress-operator-controller-manager-7cd7b55b47-25c2m   2/2     Running   0          6h5m    172.18.106.6    srini-virtual-machine   <none>           <none>
egress-operator-system   github-7c894d8d9d-9skp4                               1/1     Running   0          6h4m    172.18.106.7    srini-virtual-machine   <none>           <none>
test                     nginx                                                 1/1     Running   0          6h4m    172.18.106.8    srini-virtual-machine   <none>           <none>

kubectl get svc -A
egress-operator-system   egress-operator-controller-manager-metrics-service   ClusterIP   10.43.241.184   <none>        8443/TCP                 5h51m
egress-operator-system   github                                               ClusterIP   **10.43.243.74**    <none>        443/TCP                  5h51m

externalServicegit.yaml

apiVersion: egress.monzo.com/v1
kind: ExternalService
metadata:
  name: github
spec:
  dnsName: github.com
  hijackDns: true
  ports:
  - port: 443
    protocol: TCP
  minReplicas: 1
  maxReplicas: 1

testPod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  namespace: test
  labels:
    egress.monzo.com/allowed-github: "true"
spec:
  containers:
  - image: nginx:1.14.2
    command:
      - "sleep"
      - "604800"
    imagePullPolicy: IfNotPresent
    name: nginx
  restartPolicy: Always

Guys i was able to fix this issue as it was an issue with Networkpolicy "egress-operator-public-egress" which was blocking the egress calls from Monzo Gress gateway service. I am now allowing all the Egress calls which goes through Monzo Egress gateway and to block the other domain calls i am applying deny all network policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.k8s.io/v1","kind":"NetworkPolicy","metadata":{"annotations":{},"name":"egress-operator-public-egress","namespace":"egress-operator-system"},"spec":{"egress":
  creationTimestamp: "2021-08-30T19:14:24Z"
  generation: 2
  name: egress-operator-public-egress
  namespace: egress-operator-system
  resourceVersion: "12218"
  uid: 26fc1787-1cc7-46ab-9f29-bfd2612f7160
spec:
  egress:
  - {}
  podSelector:
    matchLabels:
      app: egress-gateway
  policyTypes:
  - Ingress
  - Egress

Default Deny All for all the other domain to be blocked

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-external-egress
  namespace: edge-system
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        # ensure your internal IP range is allowed here
        # traffic to external IPs will not be allowed from this namespace.
        # therefore, pods will have to use egress gateways
        cidr: 172.0.0.0/8 #(update your CIDR here )

This issue is fixed