natureLanguageQing / grpc-bert4keras-serve

基于grpc的bert4keras服务部署项目

grpc bert4keras serve

requirements

grpclib 
protobuf

grpclib 简介 一款实现asyncio的python的grpc扩展库 asyncio 简介 被用作多个提供高性能 Python 异步框架的基础,包括网络和网站服务,数据库连接库,分布式任务队列等等。

以文本分类项目为例子

本教程运行目录为example/text_classification

第一步：通过text_classification_proto_code_gen.sh根据pb文件生成text_classification_grpc.py和text_classification_pb2.py

第二步：编写serve层代码

import asyncio

from grpclib.server import Server, Stream
from grpclib.utils import graceful_exit

from text_classification_grpc import GreeterBase
# generated by protoc
from text_classification_pb2 import TextClassifyRequest, TextClassifyReply


class Greeter(GreeterBase):

    async def SayTextClassify(self, stream: Stream[TextClassifyRequest, TextClassifyReply]) -> None:
        request = await stream.recv_message()
        assert request is not None
        message = f'Hello, {request.text}!'
        a = TextClassifyReply.Scores(negative=1.3, neutral=0.4, positive=0.8)
        await stream.send_message(TextClassifyReply(label=message, scores=a))


async def main(*, host: str = '127.0.0.1', port: int = 50052) -> None:
    server = Server([Greeter()])
    # Note: graceful_exit isn't supported in Windows
    with graceful_exit([server]):
        await server.start(host, port)
        print(f'Serving on {host}:{port}')
        await server.wait_closed()


if __name__ == '__main__':
    asyncio.run(main())

第四步启动grpc服务的serve层

sh text_classification_serve.sh

第五步 启动grpc客户端

sh text_classification_client.sh

测试结果 100%|█████████████████████████████████████████████████████| 1000/1000 [00:14<00:00, 70.53it/s] 显卡占用率

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44       Driver Version: 495.44       CUDA Version: 11.5     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA TITAN Xp     On   | 00000000:05:00.0 Off |                  N/A |
| 23%   43C    P2    72W / 250W |   7179MiB / 12196MiB |     17%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

测试环境 GPU TITAN Xp CPU 7核 Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz