ckadner / rest-proxy

KServe V2 Protocol Rest API Implementation Proxy

KServe V2 REST Proxy

This REST Proxy leverages gRPC-Gateway to create a reverse-proxy server which translates a RESTful HTTP API into gRPC. This allows sending inference requests using the KServe V2 REST Predict Protocol to platforms that expect the gRPC V2 Predict Protocol.

Note: This is currently a work in progress, and is subject to performance and usability issues.

Generate grpc-gateway stubs

protoc -I . --grpc-gateway_out ./gen/ --grpc-gateway_opt logtostderr=true --grpc-gateway_opt paths=source_relative grpc_predict_v2.proto

Build Docker image

docker build -t kserve/rest-proxy:latest .