Kmesh does not startup properly when deployed using a locally compiled image.

[tmodak27]

What happened:

• When we try to build a Kmesh image locally using the latest code from main, we get a strange error log ./start_kmesh.sh: line 21: pid: command not found
• We cannot use Kmesh pod deployed from this image to route http request.

What you expected to happen:
We expected this pid command error log to not be present, also Kmesh should have been able to route the http request.

How to reproduce it (as minimally and precisely as possible):

Step 1: Label the default namespace

kubectl label namespace default istio.io/dataplane-mode=Kmesh --overwrite
kubectl label namespace default istio-injection=unenabled --overwrit

Step 2: Deploy 2 backend services and http client

• yaml for service httpecho-a

# kubectl  apply -f service-a.yaml
apiVersion: v1
kind: Service
metadata:
  name: httpecho-a
  labels:
    app: httpecho-a
    service: httpecho-a
spec:
  ports:
  - port: 5000
    name: http
  selector:
    app: httpecho-a
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpecho-a
  labels:
    app: httpecho-a
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpecho-a
      version: v1
  template:
    metadata:
      labels:
        app: httpecho-a
        version: v1
    spec:
      containers:
      - name: httpecho-a
        image: docker.io/istio/examples-helloworld-v1
        resources:
          requests:
            cpu: "100m"
        imagePullPolicy: IfNotPresent #Always
        ports:
        - containerPort: 5000

• yaml for service httpecho-b

# kubectly apply -f service-b.yaml
apiVersion: v1
kind: Service
metadata:
  name: httpecho-b
  labels:
    app: httpecho-b
    service: httpecho-b
spec:
  ports:
  - port: 5000
    name: httpb
  selector:
    app: httpecho-b
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpecho-b
  labels:
    app: httpecho-b
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpecho-b
      version: v2
  template:
    metadata:
      labels:
        app: httpecho-b
        version: v2
    spec:
      containers:
      - name: httpecho-b
        image: docker.io/istio/examples-helloworld-v1
        resources:
          requests:
            cpu: "100m"
        imagePullPolicy: IfNotPresent #Always
        ports:
        - containerPort: 5000

• Deploy http client

kubectl run --image=hwchiu/netutils netutils

Step 3: Deploy Virtual Service to route all requests for httpecho-b to httpecho-a

• yaml for virtualservice

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: route-b-to-a
spec:
  hosts:
  - httpecho-b
  http:
  - match:
    - port: 5000
    route:
    - destination:
        host: httpecho-a.default.svc.cluster.local
        port:
          number: 5000

Step 4: Build Kmesh image

Run make docker from the root of the kmesh repo.

Step 5: Deploy Kmesh

• In deploy/helm/values.yaml, set the correct repo and tag for the newly created image.
• Run helm install kmesh ./deploy/helm -n kmesh-system --create-namespace

Step 6: View startup logs and test routing

• In the startup logs, you can see a pid: command not found error on the first line.

[root@host-192-168-0-239 ~]# kubectl logs kmesh-sdbbg -f -n kmesh-system
./start_kmesh.sh: line 21: pid: command not found
time="2024-03-13T18:11:19Z" level=info msg="service node sidecar~10.244.0.103~kmesh-system.kmesh-system~kmesh-system.svc.cluster.local connect to discovery address istiod.istio-system.svc:15012" subsys=controller/config
time="2024-03-13T18:11:19Z" level=info msg="options InitDaemonConfig successful" subsys=manager
time="2024-03-13T18:11:20Z" level=info msg="bpf Start successful" subsys=manager
time="2024-03-13T18:11:20Z" level=info msg="controller Start successful" subsys=manager
time="2024-03-13T18:11:20Z" level=info msg="dump StartServer successful" subsys=manager
time="2024-03-13T18:11:20Z" level=info msg="start write CNI config\n" subsys="cni installer"
time="2024-03-13T18:11:20Z" level=info msg="kmesh cni use chained\n" subsys="cni installer"
time="2024-03-13T18:11:20Z" level=info msg="Copied /usr/bin/kmesh-cni to /opt/cni/bin." subsys="cni installer"
time="2024-03-13T18:11:20Z" level=info msg="kubeconfig either does not exist or is out of date, writing a new one" subsys="cni installer"
time="2024-03-13T18:11:20Z" level=info msg="wrote kubeconfig file /etc/cni/net.d/kmesh-cni-kubeconfig" subsys="cni installer"
time="2024-03-13T18:11:20Z" level=info msg="command Start cni successful" subsys=manager

• Send the http request per the below command. From the response, you can see that the request has been routed to service httpecho-b,and not httpecho-a.

$ kubectl exec -it netutils -- curl httpecho-b:5000
Hello V1, routed from pod httpecho-b-t7wx5

• Exec into the kmesh pod and run bpftool prog tracelog. You will notice no logs generated meaning Kmesh is not handling the routing for the request.

Additional info:

Environment:
Kmesh: Main branch.
OS: OpenEuler 23.03
Istio: 1.19
Kubernetes: 1.27

[Tiger Xu / Zhonghu Xu]

Sorry it was introduced recently ,let me fix it

[nlgwcy]

@tmodak27 #174