Unexpected input data type
mnts-i opened this issue · comments
I am trying to run the /extract endpoint but it always throws me the same error. The Docker image was built successfully and other API endpoints like /draw_detections run fine (I pass the images as base64 strings). Only the /extract endpoint fails. I am using the default glintr100 recognition model
OS: Windows 10 (No GPU - Using the deploy_cpu.sh's env variables)
insightface_1 | [15:32:00] INFO - Detector started
insightface_1 | [15:32:00] INFO - Warming up face detection ONNX Runtime engine...
insightface_1 | [15:32:02] INFO - Warming up ArcFace ONNX Runtime engine...
insightface_1 | [15:32:03] INFO - Warming up GenderAge ONNX Runtime engine...
insightface_1 | [2022-06-30 15:32:03 +0000] [19] [INFO] Started server process [19]
insightface_1 | [2022-06-30 15:32:03 +0000] [19] [INFO] Waiting for application startup.
insightface_1 | [2022-06-30 15:32:03 +0000] [19] [INFO] Application startup complete.
insightface_1 | [2022-06-30 15:32:14 +0000] [19] [ERROR] Exception in ASGI application
insightface_1 | Traceback (most recent call last):
insightface_1 | File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
insightface_1 | result = await app(self.scope, self.receive, self.send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
insightface_1 | return await self.app(scope, receive, send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 269, in __call__
insightface_1 | await super().__call__(scope, receive, send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 124, in __call__
insightface_1 | await self.middleware_stack(scope, receive, send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
insightface_1 | raise exc
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
insightface_1 | await self.app(scope, receive, _send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 93, in __call__
insightface_1 | raise exc
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
insightface_1 | await self.app(scope, receive, sender)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
insightface_1 | raise e
insightface_1 | File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
insightface_1 | await self.app(scope, receive, send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 670, in __call__
insightface_1 | await route.handle(scope, receive, send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 266, in handle
insightface_1 | await self.app(scope, receive, send)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 65, in app
insightface_1 | response = await func(request)
insightface_1 | File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 227, in app
insightface_1 | raw_response = await run_endpoint_function(
insightface_1 | File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
insightface_1 | return await dependant.call(**values)
insightface_1 | File "/app/app.py", line 80, in extract
insightface_1 | output = await processing.extract(images, max_size=data.max_size, return_face_data=data.return_face_data,
insightface_1 | File "/app/modules/processing.py", line 195, in extract
insightface_1 | output = await self.embed(images, max_size=max_size, return_face_data=return_face_data, threshold=threshold,
insightface_1 | File "/app/modules/processing.py", line 142, in embed
insightface_1 | faces_by_img = (e for e in await _get([img for img in imgs_iterable]))
insightface_1 | File "/app/modules/face_model.py", line 278, in get
insightface_1 | faces = list(self.process_faces(faces,
insightface_1 | File "/app/modules/face_model.py", line 149, in process_faces
insightface_1 | ga = self.ga_model.get(crops)
insightface_1 | File "/app/modules/model_zoo/exec_backends/onnxrt_backend.py", line 65, in get
insightface_1 | ret = self.rec_model.run(self.outputs, {self.input.name: imgs})[0]
insightface_1 | File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 192, in run
insightface_1 | return self._sess.run(output_names, input_feed, run_options)
insightface_1 | onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. Actual: (tensor(uint8)) , expected: (tensor(float))
Have you changed anything in deploy_cpu.sh?
Does /extract endpoint work with test data provided in swagger UI?
I have just checked it on ubuntu, everything seems to be working.
I don't run the deploy_cpu.sh script since my OS is Windows 10. Instead, I took all the env defined in the deploy_cpu script and created a docker-compose file in order to build and run the docker image (pretty much the same thing that deploy_cpu does - The image builds and runs successfully).
My .env file looks like this (copy-pasted all the variables definitions from the deploy_cpu script)
IMAGE='insightface-rest'
TAG='v0.7.3.0-cpu'
# Change InsightFace-REST logging level (DEBUG,INFO,WARNING,ERROR)
log_level=INFO
# When starting multiple containers this will be port assigned to first container
START_PORT=18081
# Set how many app instances you want to run per GPU, ensure you have enough GPU
# memory for desired number. Try running with n_workers=1 to estimate memory consumption
# per instance.
# Take note: larger number won't speed up single image inference time, it'll increase
# concurrent throughput.
n_workers=1
# Maximum image size (W,H). If your input images has fixed image size set this
# value proportional or equal to it. Otherwise select value based on your
# performance/accuracy needs.
# If input images may have both album/portrait orientation it's recommended to
# set square dimensions, like 640x640 for better accuracy.
# ATTENTION: For TensorRT backend this size currently can't be set during
# runtime.
max_size=1280,1280
# DET MODELS:
## retinaface_mnet025_v1, retinaface_mnet025_v2, retinaface_r50_v1, centerface
## scrfd_500m_bnkps, scrfd_2.5g_bnkps, scrfd_10g_bnkps
## scrfd_500m_gnkps, scrfd_2.5g_gnkps, scrfd_10g_gnkps
## yolov5l-face, yolov5m-face, yolov5s-face, yolov5n-face, yolov5n-0.5
## Note: SCRFD family models requires input image shape dividable by 32, i.e 640x640, 1024x768.
det_model=scrfd_10g_gnkps
## Maximum batch size for detection model
det_batch_size=1
# REC MODELS:
## None, arcface_r100_v1, glintr100, w600k_r50, w600k_mbf
rec_model=glintr100
## Maximum batch size for recognition model (this value also applies for GA and mask detection models)
rec_batch_size=1
# Mask detection models
## None, mask_detector, mask_detector112
mask_detector=None
# GENDER/AGE MODELS:
## None, genderage_v1
ga_model=genderage_v1
# Default settings for inference requests, can be overridden inside
# request body.
## Return base64 encoded face crops.
return_face_data=True
## Get faces embeddings. Otherwise only bounding boxes will be returned.
extract_embeddings=True
## Estimate gender/age
detect_ga=True
##Face detection probability threshold
det_thresh=0.6
and my docker-compose file looks like this:
services:
insightface:
image: ${IMAGE}:${TAG}
build:
context: ./src
dockerfile: ./Dockerfile_cpu
healthcheck:
test: 'curl -f http://localhost:18080/info || exit 1'
interval: 1m
timeout: 10s
retries: 3
environment:
- LOG_LEVEL=${log_level}
- PYTHONUNBUFFERED=0
- PORT=18080
- NUM_WORKERS=${n_workers}
- INFERENCE_BACKEND=onnx
- DET_NAME=${det_model}
- DET_THRESH=${det_thresh}
- REC_NAME=${rec_model}
- REC_IGNORE=${rec_ignore}
- MASK_DETECTOR=${mask_detector}
- MASK_IGNORE=${mask_ignore}
- GA_NAME=${ga_model}
- GA_IGNORE=${ga_ignore}
- KEEP_ALL=True
- MAX_SIZE=${max_size}
- DEF_RETURN_FACE_DATA=${return_face_data}
- DEF_EXTRACT_EMBEDDING=${extract_embeddings}
- DEF_EXTRACT_GA=${detect_ga}
- DEF_API_VER='2'
volumes:
- ./models:/models
- ./src/api_trt:/app
ports:
- ${START_PORT}:18080
I use the exact same environment variables and volumes like the script. As I mentioned before the other endpoints work well (with the same base64 string - so I don't think it's the data's fault). Only the /extract endpoint (I assume the recognition model) throws this error.
Hm, I can't reproduce your error on Linux.
Are you getting this errors with any image?
are you using WSL2 for running container? If not, try using it, I had reports earlier that CPU version works as expected on WSL2
Yeap I am using WSL2. I decided to follow the stacktrace of the error and modify the line 61 of the file /modules/model_zoo/exec_backends/onnxrt_backend.py to give the script the type it "complains" about
I changed:
imgs = np.transpose(imgs, (0, 3, 1, 2))
into
imgs = np.transpose(imgs, (0, 3, 1, 2)).astype(np.float32)
and it worked fine!! No console errors and the response object seems fine. I don't know if it's a correct solution or it messes any data up (like the face embedding vector - My Python knowledge is pretty limited almost non-existent)
Oh, I totally missed you have enabled gender/age model, which seems to be source of this error.
Thanks, I'll fix it shortly.
P.S. Actually, at closer look you have changed quiet a lot default settings, which is a bit opposite to:
I use the exact same environment variables
Applied fix, should be working now out of the box