SeldonIO / MLServer

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more

https://mlserver.readthedocs.io/en/latest/

SeldonIO/MLServer Issues

Adaptive Batching is enabled but not supported for inference streaming.
Closed 7 days ago10
Cannot import "Conversation"
Updated 9 days ago
Question - Toml restriction to fastapi patch version
Closed 9 days ago1
Adaptive batching batch timeout fires sooner than configured
Updated 9 days ago3
Python 3.12 support
Updated 11 days ago5
pydantic and pydantic-settings dependency versions on PyPI not consistent with pyproject.toml in repo
Closed 20 days ago2
Add support for nan values in PandasCodec
Updated a month ago7
Strict Version Dependency of Pydantic
Closed a month ago1
Can't specify params when running inference on MLServer
Updated a month ago1
PandasCodec.encode_request can not handle missing values
Closed 5 months ago2
The `isnan` method doesn't work on Numpy arrays with non-numeric
Updated a month ago3
Streaming model docker not working
Updated 2 months ago
GPU support for custom inference runtimes in MLServer
Updated 3 months ago
Python client library for mlflow server
Updated 3 months ago
gRPC async server looses track of `_futures`
Updated 3 months ago3
[BUG]: Kafka Message cannot be serialized to JSON when dict contains bytes
Closed 5 months ago3
mlserver-catboost not on pypi
Closed 5 months ago2
Can not import tritonclient
Closed 5 months ago1
mlflow model cannot be loaded
Closed 5 months ago8
method to_metadata_tensor gets wrong input.
Updated 5 months ago
Disabling adaptive batching leads to slower batched requests
Updated 5 months ago
pytest-lazy-fixture in dependencies
Closed 5 months ago
MLFlow model with Array type in signature not loading
Updated 5 months ago
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package
Updated 6 months ago5
Schema enforcement error when using `PandasCodec` inference request
Updated 6 months ago3
Add support for Pydantic V2
Closed 6 months ago10
Unable to use `tritonclient.grpc`
Updated 6 months ago4
Dockerfile has a conda version which is incompatible with Conda Lib Mamba
Updated 7 months ago
Shape in custom json example incorrect
Updated 7 months ago
Docs specify Pandas DataFrame Codec input shapes incorrectly.
Closed 7 months ago
Does MLServer support Skops?
Updated 7 months ago
Inference parameters are not passed to the predict function in MLFlow runtime
Updated 8 months ago
where InferenceRequest init?
Updated 8 months ago
Select GPU to be used for each worker on parallel inference
Updated 9 months ago
Adaptive batching leads to parameters being cut off
Updated 9 months ago5
Can't get metrics: `ValueError: env prometheus_multiproc_dir is not set or not a directory`
Closed 9 months ago1
`mlserver build` fails if requirements.txt includes a git path
Updated 10 months ago
InferenceRequests with Tensor of Tensors does not decode properly
Updated 10 months ago
ML Server with GRPC
Updated a year ago
gRPC fails with inferred f16 numpy array
Updated a year ago1
Examples/clarification over using OpenTelemetry tracing in MLServer
Updated a year ago
Install `fugashi`, `unidic`, `unidic-lite`, and `ipadic` as dependencies to MLServer HuggingFace to support hosting Japanese language models
Updated a year ago3
Ability to serve swagger doc dependencies from the mlserver instance or use a custom CDN
Updated a year ago
Mlserver timeout for long requests
Closed a year ago2
Python version mismatch after loading a Conda env
Closed a year ago1
OpenNMT CTranslate2 Support?
Closed a year ago2
MLServer > 1.1.0 cannot be installed on a Mac
Closed a year ago1
InferencePool should respawn dead workers
Closed a year ago1
Add support for `CatBoostRegressor`
Updated a year ago
MLServer Huggingface example fails
Closed a year ago1