Found the correct numa nodeId in the NUMANodeList
283713406 opened this issue · comments
Area
- Scheduler
- Controller
- Helm Chart
- Documents
Other components
No response
What happened?
func combineResources(numaNodes NUMANodeList, combination []int) v1.ResourceList {
resources := v1.ResourceList{}
for _, nodeIndex := range combination {
for resource, quantity := range numaNodes[nodeIndex].Resources {
if value, ok := resources[resource]; ok {
value.Add(quantity)
resources[resource] = value
continue
}
resources[resource] = quantity
}
}
return resources
}
According to the NUMANodeList saved by RNT, the NUMA nodes may not be based on node-0, node-1, node-2, This order is arranged in a random order. The subscripts in numaNodeCombination are arranged in order, and the calculated Distance value is incorrect. Therefore, it is necessary to first find the NUMA node in the numeraNodeCombination subscript corresponding to the NUMANode in the NUMANodeList, so that the calculated distance is correct.
What did you expect to happen?
{
description: "8 NUMA node optimal distance",
numaNodes: NUMANodeList{
{
NUMAID: 0,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 10, 1: 20, 2: 40, 3: 30, 4: 20, 5: 30, 6: 50, 7: 40,
},
},
{
NUMAID: 3,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 30, 1: 40, 2: 20, 3: 10, 4: 30, 5: 20, 6: 40, 7: 50,
},
},
{
NUMAID: 5,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 30, 1: 20, 2: 50, 3: 20, 4: 50, 5: 10, 6: 50, 7: 40,
},
},
{
NUMAID: 7,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 40, 1: 50, 2: 30, 3: 50, 4: 20, 5: 40, 6: 30, 7: 10,
},
},
{
NUMAID: 1,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 20, 1: 10, 2: 30, 3: 40, 4: 50, 5: 20, 6: 40, 7: 50,
},
},
{
NUMAID: 6,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 50, 1: 40, 2: 20, 3: 40, 4: 30, 5: 50, 6: 10, 7: 30,
},
},
{
NUMAID: 2,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 40, 1: 30, 2: 10, 3: 20, 4: 40, 5: 50, 6: 20, 7: 30,
},
},
{
NUMAID: 4,
Resources: v1.ResourceList{
gpuResource: resource.MustParse("1"),
v1.ResourceCPU: *resource.NewQuantity(4, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("5Gi"),
},
Costs: map[int]int{
0: 20, 1: 50, 2: 40, 3: 30, 4: 10, 5: 50, 6: 30, 7: 20,
},
},
},
podResources: v1.ResourceList{
v1.ResourceCPU: *resource.NewQuantity(3, resource.DecimalSI),
v1.ResourceMemory: resource.MustParse("2Gi"),
gpuResource: resource.MustParse("3"),
},
node: node,
expectedBitmask: NewTestBitmask(0, 1, 5),
expectedErr: nil,
expectedMinDistance: true,
},
For example, when NUMANodeList and Resources are listed above, the combination that meets the requirements and has the smallest distance should be [01 5], and the resulting combination is indeed [01 4]
How can we reproduce it (as minimally and precisely as possible)?
No response
Anything else we need to know?
No response
Kubernetes version
[root@master1 ~]# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.13", GitCommit:"49433308be5b958856b6949df02b716e0a7cf0a3", GitTreeState:"clean", BuildDate:"2023-04-12T12:15:50Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"linux/arm64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.13", GitCommit:"49433308be5b958856b6949df02b716e0a7cf0a3", GitTreeState:"clean", BuildDate:"2023-04-12T12:08:36Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"linux/arm64"}
Scheduler Plugins version
master
thanks for the report. Did you notice this bug operating the plugin or was it noticed reviewing the code?
thanks for the report. Did you notice this bug operating the plugin or was it noticed reviewing the code?
Thank you for your reply. I discovered this issue during testing by looking at the logs. When ScoringStrategityType is LeastNUMANodes, it will find the required resources for Pod, and of course, only numaNodes. Count() needs to be calculated during the final scoring, so even if an incorrect suitableCombination is obtained, it will not affect it. But it is indeed possible to calculate an incorrect suitableCombination here
thanks for the report. Did you notice this bug operating the plugin or was it noticed reviewing the code?
Thank you for your reply. I discovered this issue during testing by looking at the logs. When ScoringStrategityType is LeastNUMANodes, it will find the required resources for Pod, and of course, only numaNodes. Count() needs to be calculated during the final scoring, so even if an incorrect suitableCombination is obtained, it will not affect it. But it is indeed possible to calculate an incorrect suitableCombination here
Thanks, makes sense