Model per instance model-mesh by default
fsatka opened this issue · comments
Now model load only on one instance, and lazy loading on another pods, when reauest has come.
Can we modify internal modelmesh parameters for default loading model on all ServingRuntime instances?
@fsatka -- ModelMesh was designed to optimize resource utilization. Why would you want to load additional instances of the same model/predictor/ISVC on all serving runtime pods regardless of inference request traffic? Just for testing purposes?