Giters

kubernetes-sigs / scheduler-plugins

Repository for out-of-tree scheduler plugins based on scheduler framework.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Multiple informers for the same resource(NodeResourceTopologies) may be serving at the same time

NoicFank opened this issue · comments

commented

Area

  • Scheduler
  • Controller
  • Helm Chart
  • Documents

Other components

No response

What happened?

Why could this happen?

Once we configure plugin NodeResourceTopologyMatch in multi profiles(must more than one), then informers about NodeResourceTopologies may start multi times with different ShardInformerFactory.

Since informer about NodeResourceTopologies is created within this plugin's func New() here as following:

func New(args runtime.Object, handle framework.Handle) (framework.Plugin, error) {
	...
	nrtCache, err := initNodeTopologyInformer(tcfg, handle)
	...

And, informer about NodeResourceTopologies created from the brandNew Factory within func initNodeTopologyInformer here, as following:

func initNodeTopologyInformer(tcfg *apiconfig.NodeResourceTopologyMatchArgs, handle framework.Handle) (nrtcache.Interface, error) {
	...
	topologyInformerFactory := topologyinformers.NewSharedInformerFactory(topoClient, 0)
	nodeTopologyInformer := topologyInformerFactory.Topology().V1alpha2().NodeResourceTopologies()
	nodeTopologyLister := nodeTopologyInformer.Lister()
        ...
}

Then, When we create scheduler profile one bye one, The func New() would be called many times, since plugin NodeResourceTopologyMatch exists in multiple profiles. Then multiple informers about NodeResourceTopologies will be produced.

What are the negative effects?

  • Data inconsistencies among those informers, even if the time difference is small.
  • Waste of local storage: as multiple same indexer with the same content will exist

What did you expect to happen?

How to prevent it?

Creating NewSharedInformerFactory When we create Scheduler instead of creating plugins. And we pass the created Informers about CRD from NewSharedInformerFactory to those needed plugins while it started.

How can we reproduce it (as minimally and precisely as possible)?

No response

Anything else we need to know?

@Huang-Wei Please see if this issue makes sense, thanks.

Kubernetes version

$ kubectl version
# paste output here

Scheduler Plugins version

commented

This is a rough implementation to better state the solution. #657

@Huang-Wei PTAL, thanks.

commented

Please see if this issue makes sense, thanks.

Yes, the issue is generic for plugins that are registered in multiple profiles but want to instantiate particular informers once in New().

commented

Please see if this issue makes sense, thanks.

Yes, the issue is generic for plugins that are registered in multiple profiles but want to instantiate particular informers once in New().

Thank you for your speedy reply!

It's always a pleasure to ask you questions.

commented

PR #655 will fix this.

details:
#657 (comment)

commented

@NoicFank: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.