kubeflow / fairing

Python SDK for building, training, and deploying ML models

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Fairing Serving job overwrite the user custom pod container command

songm28 opened this issue · comments

/kind bug

What steps did you take and what happened:
I set the command 'python xxx.py' for the pod in the preprocessor to build and deploy the docker image, but the command is overwritten by 'seldon-core-microservice xxx REST --service-type=MODEL --persistence=0' after deploy the fairing serving job.

What did you expect to happen:
the expectation is to run both the custom command and the .seldon-core-microservice command, not just run seldon-core-microservice.
Maybe we can try to use 'sh -c command1 && command 2' to set the pod command when deploying the serving job.

Anything else you would like to add:
The code to change maybe here: [https://github.com/kubeflow/fairing/blob/master/kubeflow/fairing/deployers/serving/serving.py]
def deploy(self, pod_spec):
"""deploy a seldon-core REST service
:param pod_spec: pod spec for the service
"""
self.job_id = str(uuid.uuid1())
self.labels['fairing-id'] = self.job_id
for fn in self.pod_spec_mutators:
fn(self.backend, pod_spec, self.namespace)
pod_template_spec = self.generate_pod_template_spec(pod_spec)
pod_template_spec.spec.containers[0].command = ["seldon-core-microservice",
self.serving_class, "REST",
"--service-type=MODEL", "--persistence=0"]

self.deployment_spec = self.generate_deployment_spec(pod_template_spec)
self.service_spec = self.generate_service_spec()

Environment:

  • Fairing version: (use python -c "import kubeflow.fairing; print(kubeflow.fairing.__version__)"): 0.7.2
  • Kubeflow version: (version number can be found at the bottom left corner of the Kubeflow dashboard): dev_local| Dashboard:v0.0.2 | Isolation-Mode: single-user
  • Minikube version:
  • Kubernetes version: (use kubectl version): v1.12.7-rancher1
  • OS (e.g. from /etc/os-release):

NOTE: If you are using fair from master, please provide us the git commit hash.

Issue Label Bot is not confident enough to auto-label this issue.
See dashboard for more details.