tensorflow_serving.apis doesn't work in Docker

[Ihar]

Bug Report

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 22

# requirements.txt
fastapi
uvicorn
requests
tensorflow-serving-api
grpcio

# Dockerfile

FROM python:3.11

# set work directory
WORKDIR /app

# copy project
COPY . .

# set env variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install dependencies
RUN pip install -r requirements.txt

from tensorflow_serving.apis import model_service_pb2_grpc
from tensorflow_serving.apis import model_management_pb2
from tensorflow_serving.config import model_server_config_pb2

Describe the problem

#error

root@a2e3916f3774:/app# python main.py 
2023-03-30 09:00:32.199385: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-03-30 09:00:32.200511: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-03-30 09:00:32.218450: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-03-30 09:00:32.218668: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-03-30 09:00:32.674915: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
  File "/app/main.py", line 5, in <module>
    from tensorflow_serving.apis import model_service_pb2_grpc
  File "/usr/local/lib/python3.11/site-packages/tensorflow_serving/apis/model_service_pb2_grpc.py", line 22, in <module>
    from tensorflow_serving.apis import get_model_status_pb2 as tensorflow__serving_dot_apis_dot_get__model__status__pb2
  File "/usr/local/lib/python3.11/site-packages/tensorflow_serving/apis/get_model_status_pb2.py", line 17, in <module>
    from tensorflow_serving.apis import status_pb2 as tensorflow__serving_dot_apis_dot_status__pb2
  File "/usr/local/lib/python3.11/site-packages/tensorflow_serving/apis/status_pb2.py", line 68, in <module>
    _STATUSPROTO.fields_by_name['error_code'].enum_type = tensorflow_dot_core_dot_protobuf_dot_error__codes__pb2._CODE
                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'tensorflow.core.protobuf.error_codes_pb2' has no attribute '_CODE'

[Niraj Singh]

@IgorHoholko,

Which release of tensorflow-serving-api you are using? Can you please make sure that you are on 2.11.1 latest release?

Also, I can see you are missing tensorflow in your requirements.txt. Please make sure to add tensorflow in requirements file and follow this article on to serve model on TF Serving.

Thank you!

[Ihar]

@singhniraj08

I had this:

protobuf                     4.22.1
tensorflow                   2.12.0
tensorflow-estimator         2.12.0
tensorflow-hub               0.13.0
tensorflow-io-gcs-filesystem 0.32.0
tensorflow-serving-api       2.10.1

When I specify tensorflow-serving-api==2.11.1:

# requirements.txt
fastapi
uvicorn
requests
tensorflow-serving-api==2.11.1
numpy
grpcio

I am getting this error:

#0 22.75 INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
#0 22.75 INFO: pip is looking at multiple versions of tensorflow-serving-api to determine which version is compatible with other requirements. This could take a while.
#0 22.75 ERROR: Cannot install -r requirements.txt (line 4) and tensorflow-serving-api because these package versions have conflicting dependencies.
#0 22.75 
#0 22.75 The conflict is caused by:
#0 22.75     tensorflow-serving-api 2.11.1 depends on protobuf<3.20 and >=3.9.2
#0 22.75     tensorflow 2.12.0 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <5.0.0dev and >=3.20.3
#0 22.75 
#0 22.75 To fix this you could try to:
#0 22.75 1. loosen the range of package versions you've specified
#0 22.75 2. remove package versions to allow pip attempt to solve the dependency conflict
#0 22.75 
#0 22.75 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
#0 22.75 

I tried to fix version:

# requirements.txt
protobuf==3.9.2
tensorflow==2.12.0
tensorflow-serving-api==2.11.1
fastapi
uvicorn
requests
numpy
grpcio

Now I am getting this error:

#0 17.34 INFO: pip is looking at multiple versions of protobuf to determine which version is compatible with other requirements. This could take a while.
#0 17.34 ERROR: Cannot install -r requirements.txt (line 2) and protobuf==3.9.2 because these package versions have conflicting dependencies.
#0 17.34 
#0 17.34 The conflict is caused by:
#0 17.34     The user requested protobuf==3.9.2
#0 17.34     tensorflow 2.12.0 depends on protobuf!=4.21.0, !=4.21.1, !=4.21.2, !=4.21.3, !=4.21.4, !=4.21.5, <5.0.0dev and >=3.20.3
#0 17.34 
#0 17.34 To fix this you could try to:
#0 17.34 1. loosen the range of package versions you've specified
#0 17.34 2. remove package versions to allow pip attempt to solve the dependency conflict
#0 17.34 
#0 17.34 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Could you please send me the configuration which will work for sure ?:)

[Niraj Singh]

@IgorHoholko,

Below versions would resolve the protobuf dependency conflicts. Please try and let us know if that works. Thank you!

tensorflow-serving-api-2.11.1
tensorflow-2.11.1
protobuf-3.19.6

[Ihar]

Thank you.
I used your requirements, that helped when I changed python:3.11 to python:3.9.
With python:3.11 doesn't work.