kmesh-net / kmesh

High Performance ServiceMesh Data Plane Based on Programmable Kernel

Home Page:https://kmesh.net

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add support for profiling kmesh controlplane

hzxuzhonghu opened this issue · comments

What would you like to be added:

Recently we have figured out some issues on perf, stability, i have a rough though, we can add some metrics to help profiling kmesh-daemon. Like

  1. Basic CPU/Memory usage
  2. xDS related metrics like xds response handling time., bpf map operations

Why is this needed:

Currently, kmesh-daemon is like a black box, we can see some xds configurations by dumping the bpf map. But we couldn't know whether it is consistent with xds response, which flow is slow, which object is memory costing, etc. This is super critical for production use.

we can use net/http/pprof package

xDS related metrics like xds response handling time., bpf map operations

Shall we use Prometheus to expose custom metrics?

we can split some tasks, metrics can be one

pprof will start a http server, shall we make the port configurable or use a default one?

seems that pprof also supports custom profile

/assign