greamown / pytorch-resnet18-service-using-websocket

Pytorch-resnet18-service-using-websocket

We are using PyTorch ResNet18, converted to ONNX, to perform inference on images. The communication method being used is the WebSocket function.

Getting Started

Windows

Pre-requirements

nvidia-driver

• Tutorial-nvidia-driver

WSL

• Tutorial-WSL

DOCKER DESKTOP

• Tutorial-docker-desktop

NGINX Because WLS and the local Windows service have different IP addresses, you need to install nginx to proxy the IP.

• Tutorial-Nginx
• Tutorial-nginx.conf

Ubuntu

Pre-requirements

Install nvidia-driver, nvidia-docker and docker before installing the docker container.

• Tutorial-nvidia-driver

• Tutorial-docker

• Tutorial-nvidia-docker

• Add docker to sudo group

  • Tutourial

  sudo groupadd docker
  sudo usermod -aG docker $USER
  sudo chmod 777 /var/run/docker.sock
  

Building image

sudo chmod u+x ./docker/*.sh
sudo ./docker/build.sh

Run docker container

sudo chmod u+x ./docker/*.sh
sudo ./docker/run.sh

Run webAPI service

gunicorn -b 0.0.0.0:5000 --workers 1 --threads 100 webapi:app

• If you want to change port numbers, you shold change 0.0.0.0:5000.

Testing

python3 client/client_ws_100pics.py -p <folder/files>
python3 client/client_ws_batch_pickle.py -p <folder/files> --batch 8

• p: The path of inference images. you can input the folder path or the image path.

• If an error occurs with the WebSocket's create_connection function, you should do the following:

  pip uninstall websocket
  pip uninstall websocket-client
  pip install websocket
  pip install websocket-client

Reference

• flask-sock
  • https://github.com/miguelgrinberg/flask-sock