gitathrun / bert_qa_api_base_image

Huggineface BERT-QA API base Image

Author: Teng Fu

AIM

It is for Bert-QA development purpose, it consists following feature:

• fastAPI, gunicorn and uvicorn for API development

• PyTorch 1.6.0

• Cuda 10.1 and cuDNN 7

• Huggingface Transformer library

This allows huggingface BERT models able to:

• execute with GPU-enabled

• provide REST API service through fastAPI

Base Images

This image is based on:

• tiangolo/uvicorn-gunicorn:python3.8

• uvicorn-gunicorn-fastapi-docker

• pytorch/pytorch:1.6.0-cuda10.1-cudnn7-runtime

pytorch-cuda image is the main base image. In this main base image, the Python version is 3.8.5.

Usage

Make sure the nvidia driver is >=R418

sudo docker run -it --runtime=nvidia huggingface-torch-gpu-fastapi:0.1 sh