Allow custom HTTP status in mms.service.Service

[jcsaaddupuy]

Hi,

Currently, mms is only responding with https status success 200 and 503/ 507 (see mms/service.py#L107-L128).

Implementing a custom sagemaker multimodel endpoint with a custom model, it would be really usefull to be able to respond to the client with others status (for ex a 400 when wrong parameters are given) or a 404 whenever a model is not found.

Now, whenever a validation error is raise in our code, the client will only ends up with a 503 that is not really helpfull.

For the 400, it could be handled in mms/service.py with something as simple as

try:
    ret = self._entry_point(input_batch, self.context)
# ...
except ValueError as e:
    return create_predict_response(None, req_id_map, e.args[0], 400)
except Exception:
# ...

For others status codes, I'm not sure what would be the best way to do it.
Maybe being able to return a status code directly from the handler, in the flask way.

I will gladly provie a MR if this raise any interest.

[eveTu]

Did you solve this issue? I'm facing the same problem now.

[Aaqib]

PR #965 merged

[lxning]

There is a bug in the custom error code implementation. See https://github.com/awslabs/multi-model-server/blob/master/mms/service.py#L141 . The string interpolation is missing the necessary braces, so any attempts to use a custom error code give us only literally "mms.service.PredictionException: message : error_code" to work with, not the actual message and error code values