Use kubectl to install in the kind cluster, coredns prompts an error

Question

Use kubectl to install in the kind cluster, coredns prompts an error

xiaofan-linux opened this issue a year ago · comments

Describe the bug
Use kubectl to install in the kind cluster, coredns prompts an error

install

export VERSION="v2023.01.8"
kubectl apply -f "https://github.com/orchest/orchest/releases/download/${VERSION}/orchest-controller.yaml"
kubectl apply -f "https://github.com/orchest/orchest/releases/download/${VERSION}/example-orchestcluster.yaml"

status

>>> get pod --all-namespaces
NAMESPACE            NAME                                                              READY   STATUS      RESTARTS      AGE
ingress-nginx        ingress-nginx-admission-create-jrrsn                              0/1     Completed   0             5m35s
ingress-nginx        ingress-nginx-admission-patch-zr5nd                               0/1     Completed   0             5m35s
ingress-nginx        ingress-nginx-controller-5696cf8778-mpr2p                         1/1     Running     0             5m35s
kube-flannel         kube-flannel-ds-7p7qs                                             1/1     Running     0             5m39s
kube-flannel         kube-flannel-ds-9nm2j                                             1/1     Running     0             5m39s
kube-system          coredns-5d78c9869d-4d597                                          1/1     Running     0             6m6s
kube-system          coredns-5d78c9869d-rlsq6                                          1/1     Running     0             6m6s
kube-system          etcd-kind-control-plane                                           1/1     Running     0             6m18s
kube-system          kube-apiserver-kind-control-plane                                 1/1     Running     0             6m21s
kube-system          kube-controller-manager-kind-control-plane                        1/1     Running     0             6m19s
kube-system          kube-proxy-4wfzd                                                  1/1     Running     0             6m6s
kube-system          kube-proxy-gwvns                                                  1/1     Running     0             6m2s
kube-system          kube-scheduler-kind-control-plane                                 1/1     Running     0             6m18s
local-path-storage   local-path-provisioner-5d7949c7d4-z79gx                           1/1     Running     0             6m6s
orchest              auth-server-9474c8c4d-tdnmg                                       1/1     Running     0             107s
orchest              buildkit-daemon-phx45                                             1/1     Running     0             32s
orchest              celery-worker-56c895d66d-mfhcc                                    1/1     Running     0             2m6s
orchest              docker-registry-6c48fbb8f-mzkrk                                   1/1     Running     0             4m11s
orchest              node-agent-77b2g                                                  1/1     Running     0             27s
orchest              orchest-api-88c77fcc5-8cc2w                                       1/1     Running     0             3m13s
orchest              orchest-argo-workflow-argo-workflows-workflow-controller-62nv4h   1/1     Running     0             3m55s
orchest              orchest-controller-7b99dfbbc5-c2984                               1/1     Running     0             4m21s
orchest              orchest-database-9cdd4f989-x4gh4                                  1/1     Running     0             3m49s
orchest              orchest-webserver-5b5cfc4bfc-qsgsk                                2/2     Running       2 (24s ago)   74s
orchest              rabbitmq-server-7ff9b989d-jmn8w                                   1/1     Running     0             3m26s

error log

>>> logs -f -n kube-system coredns-5d78c9869d-4d597
.:53
[INFO] plugin/reload: Running configuration SHA512 = 591cf328cccc12bc490481273e738df59329c62c0b729d94e8b61db9961c2fa5f046dd37f1cf888b953814040d180f52594972691cd6ff41be96639138a43908
CoreDNS-1.10.1
linux/amd64, go1.20, 055b2c3
[ERROR] plugin/errors: 2 712500952834338557.2302118844394776856. HINFO: read udp 192.168.0.3:49837->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 712500952834338557.2302118844394776856. HINFO: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 712500952834338557.2302118844394776856. HINFO: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 712500952834338557.2302118844394776856. HINFO: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: read udp 192.168.0.3:44682->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: read udp 192.168.0.3:38786->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: read udp 192.168.0.3:41421->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: read udp 192.168.0.3:52392->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.google.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.google.internal. A: read udp 192.168.0.3:37663->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.google.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.google.internal. A: read udp 192.168.0.3:45678->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.google.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.google.internal. AAAA: read udp 192.168.0.3:41939->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.google.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.google.internal. A: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.google.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.google.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: read udp 192.168.0.3:60975->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: read udp 192.168.0.3:34319->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: read udp 192.168.0.3:35455->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: read udp 192.168.0.3:57296->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: read udp 192.168.0.3:39008->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: read udp 192.168.0.3:38017->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: read udp 192.168.0.3:44080->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.asia-east1-b.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.c.applied-city-381113.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 api.github.com.c.applied-city-381113.internal. A: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. A: read udp 10.244.1.2:54756->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. A: read udp 10.244.1.2:38175->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. AAAA: dial udp [2001:4860:4860::8844]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. A: read udp 10.244.1.2:55777->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. A: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. AAAA: read udp 10.244.1.2:58846->172.22.0.1:53: i/o timeout
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. AAAA: dial udp [2001:4860:4860::8888]:53: connect: network is unreachable
[ERROR] plugin/errors: 2 docker-registry.orchest.svc.cluster.local.google.internal. A: read udp 10.244.1.2:38492->172.22.0.1:53: i/o timeout

The cluster domain name ends with google.internal., which seems a bit unreasonable

OS (e.g. macOS): Fedora
Browser (e.g. Chrome): Chrome
Orchest's version (in the settings page): v2023.04.2

welcome · Answer 1 · Sat May 13 2023 20:04:40 GMT+0800 (China Standard Time)

❤️ Thanks for opening your first issue! We really appreciate your input.
💬 If you need to connect more synchronously with members of the Orchest community, please feel free to chat with us on our Slack.