monzo / egress-operator

A Kubernetes operator to produce egress gateway Envoy pods and control access to them with network policies

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

make run error : "no matches for kind "ExternalService" in version egress.monzo.com/v1"

lazywhite opened this issue · comments

here is the "make run" output

/root/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go run ./main.go
2020-02-29T22:53:37.829Z	INFO	controller-runtime.metrics	metrics server is starting to listen	{"addr": ":8080"}
2020-02-29T22:53:37.833Z	INFO	setup	starting manager
2020-02-29T22:53:37.837Z	INFO	controller-runtime.manager	starting metrics server	{"path": "/metrics"}
2020-02-29T22:53:37.937Z	INFO	controller-runtime.controller	Starting EventSource	{"controller": "externalservice", "source": "kind source: /, Kind="}
2020-02-29T22:53:39.306Z	ERROR	controller-runtime.source	if kind is a CRD, it should be installed before calling Start	{"kind": "ExternalService.egress.monzo.com", "error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
	/root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start
	/root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/source/source.go:88
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
	/root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:165
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start
	/root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:198
sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startLeaderElectionRunnables.func1
	/root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/manager/internal.go:477
2020-02-29T22:53:39.319Z	DEBUG	controller-runtime.manager	leader-election runnable finished	{"runnable type": "*controller.Controller"}
2020-02-29T22:53:39.319Z	ERROR	setup	problem running manager	{"error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
	/root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
main.main
	/root/egress-operator/main.go:82
runtime.main
	/usr/local/go/src/runtime/proc.go:203
exit status 1
Makefile:25: recipe for target 'run' failed
make: *** [run] Error 1

@lazywhite @dovys @rajeshsingh624: can someone let me know how this issue got resolved ? I am getting the similar issue while executing "make run" & "make deploy...." commands. Please see the below errors:

root@Ubuntu18-Virtual-Machine:~/egress-operator#  make run
go: creating new go.mod: module tmp
go get: added sigs.k8s.io/controller-tools v0.2.4
/root/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
go fmt ./...
go vet ./...
/root/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go run ./main.go
2021-07-08T18:13:38.731+0530    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2021-07-08T18:13:38.732+0530    INFO    setup   starting manager
2021-07-08T18:13:38.732+0530    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2021-07-08T18:13:38.834+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T18:13:40.978+0530    ERROR   controller-runtime.source       if kind is a CRD, it should be installed before calling Start   {"kind": "ExternalService.egress.monzo.com", "error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
        /root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/source/source.go:88
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:165
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:198
sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startLeaderElectionRunnables.func1
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/manager/internal.go:477
2021-07-08T18:13:40.978+0530    DEBUG   controller-runtime.manager      leader-election runnable finished       {"runnable type": "*controller.Controller"}
2021-07-08T18:13:40.978+0530    ERROR   setup   problem running manager {"error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
        /root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
main.main
        /root/egress-operator/main.go:89
runtime.main
        /usr/local/go/src/runtime/proc.go:225
exit status 1
Makefile:25: recipe for target 'run' failed
make: *** [run] Error 1

Note: As make was not recognized in ubuntu 18 , I have run apt-get install build-essential in order to install it.
Do I need to change any other file apart from the setup instructions . Please can you let me know what might went wrong ?

@lazywhite @dovys @rajeshsingh624: can someone let me know how this issue got resolved ? I am getting the similar issue while executing "make run" & "make deploy...." commands. Please see the below errors:

root@Ubuntu18-Virtual-Machine:~/egress-operator#  make run
go: creating new go.mod: module tmp
go get: added sigs.k8s.io/controller-tools v0.2.4
/root/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
go fmt ./...
go vet ./...
/root/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go run ./main.go
2021-07-08T18:13:38.731+0530    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2021-07-08T18:13:38.732+0530    INFO    setup   starting manager
2021-07-08T18:13:38.732+0530    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2021-07-08T18:13:38.834+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T18:13:40.978+0530    ERROR   controller-runtime.source       if kind is a CRD, it should be installed before calling Start   {"kind": "ExternalService.egress.monzo.com", "error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
        /root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/source/source.go:88
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:165
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:198
sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startLeaderElectionRunnables.func1
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/manager/internal.go:477
2021-07-08T18:13:40.978+0530    DEBUG   controller-runtime.manager      leader-election runnable finished       {"runnable type": "*controller.Controller"}
2021-07-08T18:13:40.978+0530    ERROR   setup   problem running manager {"error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
        /root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
main.main
        /root/egress-operator/main.go:89
runtime.main
        /usr/local/go/src/runtime/proc.go:225
exit status 1
Makefile:25: recipe for target 'run' failed
make: *** [run] Error 1

Note: As make was not recognized in ubuntu 18 , I have run apt-get install build-essential in order to install it.
Do I need to change any other file apart from the setup instructions . Please can you let me know what might went wrong ?

you have to run make install at the first place

@lazywhite : Just after cloning the project, I cd egress-operator/ and ran make install but it's giving error:

root@Ubuntu18-Virtual-Machine:~/egress-operator# make install

Command 'make' not found, but can be installed with:

apt install make
apt install make-guile

Do you mean apt-get install make OR apt-get install build-essential ?

@lazywhite : Just after cloning the project, I cd egress-operator/ and ran make install but it's giving error:

root@Ubuntu18-Virtual-Machine:~/egress-operator# make install

Command 'make' not found, but can be installed with:

apt install make
apt install make-guile

Do you mean apt-get install make OR apt-get install build-essential ?

execute these command
apt install make
make install

apt-get install build-essential
this command will install gcc for you

@lazywhite Now I see error error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\" . Have you edited any yaml file also before firing make run ?

root@Ubuntu18-Virtual-Machine:~/egress-operator# make run
go: creating new go.mod: module tmp
go get: added sigs.k8s.io/controller-tools v0.2.4
/root/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
go fmt ./...
go vet ./...
go: downloading github.com/onsi/ginkgo v1.8.0
go: downloading github.com/onsi/gomega v1.5.0
go: downloading sigs.k8s.io/testing_frameworks v0.1.2
go: downloading github.com/hpcloud/tail v1.0.0
go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
/root/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go run ./main.go
2021-07-08T22:10:20.212+0530    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2021-07-08T22:10:20.212+0530    INFO    setup   starting manager
2021-07-08T22:10:20.213+0530    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2021-07-08T22:10:20.313+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T22:10:22.462+0530    ERROR   controller-runtime.source       if kind is a CRD, it should be installed before calling Start   {"kind": "ExternalService.egress.monzo.com", "error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
        /root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/source/source.go:88
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:165
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:198
sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startLeaderElectionRunnables.func1
        /root/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/manager/internal.go:477
2021-07-08T22:10:22.462+0530    DEBUG   controller-runtime.manager      leader-election runnable finished       {"runnable type": "*controller.Controller"}
2021-07-08T22:10:22.462+0530    ERROR   setup   problem running manager {"error": "no matches for kind \"ExternalService\" in version \"egress.monzo.com/v1\""}
github.com/go-logr/zapr.(*zapLogger).Error
        /root/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
main.main
        /root/egress-operator/main.go:89
runtime.main
        /usr/local/go/src/runtime/proc.go:225
exit status 1
Makefile:25: recipe for target 'run' failed
make: *** [run] Error 1

did u exec make install?

did u exec make install?

@lazywhite Yes I did exec make install. Still it says Kustomize not install (though I have installed kustomize ). So I installed Kustomize again. But getting error in make install

root@Ubuntu18-Virtual-Machine:~/egress-operator# curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash

{Version:kustomize/v4.2.0 GitCommit:d53a2ad45d04b0264bcee9e19879437d851cb778 BuildDate:2021-06-30T22:49:26Z GoOs:linux GoArch:amd64}
kustomize installed to /root/egress-operator/kustomize
root@Ubuntu18-Virtual-Machine:~/egress-operator#
root@Ubuntu18-Virtual-Machine:~/egress-operator# make install
go: creating new go.mod: module tmp
go get: added sigs.k8s.io/controller-tools v0.2.4
/root/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
kustomize build config/crd | kubectl apply -f -
/bin/sh: 1: kustomize: not found
error: no objects passed to apply
Makefile:29: recipe for target 'install' failed
make: *** [install] Error 1
root@Ubuntu18-Virtual-Machine:~/egress-operator#

@lazywhite Do I need to export path (set path variable) of Kustomize ?

https://kustomize.io/
download this and put it in your PATH

@lazywhite : it worked now I guess. Does this mean that this create an ExternalService object ? The terminal has not exited by itself. So shall I continue with coredns setup steps now ?

root@Ubuntu18-Virtual-Machine:~/egress-operator# make run
go: creating new go.mod: module tmp
go get: added sigs.k8s.io/controller-tools v0.2.4
/root/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths="./..."
go fmt ./...
go vet ./...
/root/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go run ./main.go
2021-07-08T22:39:17.246+0530    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2021-07-08T22:39:17.247+0530    INFO    setup   starting manager
2021-07-08T22:39:17.247+0530    INFO    controller-runtime.manager      starting metrics server {"path": "/metrics"}
2021-07-08T22:39:17.348+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T22:39:17.449+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T22:39:17.550+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T22:39:17.651+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T22:39:17.752+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T22:39:17.852+0530    INFO    controller-runtime.controller   Starting EventSource    {"controller": "externalservice", "source": "kind source: /, Kind="}
2021-07-08T22:39:17.954+0530    INFO    controller-runtime.controller   Starting Controller     {"controller": "externalservice"}
2021-07-08T22:39:18.054+0530    INFO    controller-runtime.controller   Starting workers        {"controller": "externalservice", "worker count": 1}

yes, it is working, do as the readme said

@lazywhite : Hey have you faced any coredns pod related issue? once I added my my_azure_repo/egress-operator-coredns:latest image in coredns Deployment file, I see my coredns pod status changed from Running to ImagePullBackOff .

Coredns Pod description shows:

root@Ubuntu18-Virtual-Machine:~/egress-operator# kubectl describe pod coredns-67f9b7c578-m57p7 -n kube-system
Name:                 coredns-67f9b7c578-m57p7
Namespace:            kube-system
Priority:             2000000000
Priority Class Name:  system-cluster-critical
Node:                 Ubuntu18-Virtual-Machine/192.168.1.10
Start Time:           Fri, 09 Jul 2021 10:27:38 +0530
Labels:               k8s-app=kube-dns
                      pod-template-hash=67f9b7c578
Annotations:          cni.projectcalico.org/podIP: 172.16.216.5/32
                      cni.projectcalico.org/podIPs: 172.16.216.5/32
Status:               Pending
IP:                   172.16.216.5
IPs:
  IP:           172.16.216.5
Controlled By:  ReplicaSet/coredns-67f9b7c578
Containers:
  coredns:
    Container ID:
    Image:         my_azure_repo/egress-operator-coredns:latest
    Image ID:
    Ports:         53/UDP, 53/TCP, 9153/TCP
    Host Ports:    0/UDP, 0/TCP, 0/TCP
    Args:
      -conf
      /etc/coredns/Corefile
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Limits:
      memory:  170Mi
    Requests:
      cpu:        100m
      memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=1s period=10s #success=1 #failure=3
    Readiness:    http-get http://:8181/ready delay=0s timeout=1s period=2s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /etc/coredns from config-volume (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-sxd85 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      coredns
    Optional:  false
  kube-api-access-sxd85:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              beta.kubernetes.io/os=linux
Tolerations:                 CriticalAddonsOnly op=Exists
                             node-role.kubernetes.io/control-plane:NoSchedule op=Exists
                             node-role.kubernetes.io/master:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason   Age                    From     Message
  ----    ------   ----                   ----     -------
  Normal  BackOff  1s (x1648 over 6h15m)  kubelet  Back-off pulling image "my_azure_repo/egress-operator-coredns:latest"

You should push your image to docker registry

@lazywhite Yes I did that. I am able to pull my my_azure_repo/egress-operator-coredns:latest from same ubuntu machine. But pod is failing to pull.

@shreya-bhatnagar you should make sure all worker have coredns scheduled can access your image

@lazywhite : Hey my coredns and egress-operator-deployment pods are running fine now. But egress-operator-controller-manager pod (created after make deploy IMG=my_azure_repo/egress-operator:v0.1 ) is giving error in pod description as:

 Failed   m2s (x4 over 4m35s)   kubelet  Error: failed to create containerd task: 
 OCI runtime create failed: container_linux.go:380: starting container process 
 caused: exec: "/manager": stat /manager: no such file or directory: unknown

my make run and make deploy IMG=my_azure_repo/egress-operator:v0.1 also worked fine without any error.

You have any idea why is it so ? some path /manager not able to be recognized by makefile it seems.?