[Subtask]: Join that creates too many proc
fengttt opened this issue · comments
Parent Issue
#16617 Case 1, more info.
Detail of Subtask
Also this one consumes lots of memory.
We have too many pros. So we must answer the following questions,
- Are they necessary? After we fix Join pipeline (that is, join is one single pipeline, with one proc), all these are gone, correct?
- Should these joins run in parallel at all. If yes, how many threads/procs.
Describe implementation you've considered
No response
Additional information
No response
等tp pipeline合并完成以后再观察一下
等tp pipeline合并完成以后再观察一下
pipeline重构只能减少process的数量，要从根源上解决问题，需要在这里做个proc的复用
跑了一轮sysbench delete test， 从2.4%下降到2% 还需要继续优化
又跑了一次，现在已经下降到0.18%