solidglue / Recommender_System_Inference_Services

Large scale recommender system inference Microservices and APIs （Dubbo 、gRPC and REST ） with Golang.

Introduction

基于Goalng、Docker和微服务**实现了高并发、高性能和高可用的推荐系统推理微服务，包括多种召回/排序服务，并提供多种接口访问方式（REST、gRPC和Dubbo）等，每日可处理上千万次推理请求。
Large Scale Deep Learning Recommender Systems Inference Services / Microservices base on TFServing、Faiss 、Redis、Dubbo、Nacos、gRPC and Golang. This system provides real-time inference services(Dubbo api、gPRC api and REST api),which can withstand millions of inference requests per day.

Dependent Components

The model inference microservices based on deep learning mainly uses the following components:

Type	Component	Description
Data	Hive / Spark	ETL millions users's behavior data and then build the feature data warehouse.
	Redis	Save the training samples in TFRecord format and store them in Redis Cluster.
Model	TensorFlow	Training deep learning recall / rank model , alse you can use other deep learning framework ,but need save models as *.pb format.
	TensorFlow Serving	Deploy models and provide a grpc service.
	FAISS	Quick ANN search thousands items from millions items.
Microservices	Nacos	Manage config files and services.
	Dubbo	Build dubbo protocol RPC services and register them to Nacos.
	Hystrix	How to distribute traffic during peak traffic (Latency and Fault Tolerance).
	Skywalking	Record the time spent on each request.
Deploy	Docker	Docker containerization deployment services.
	Kubernetes	Manage dockers and monitor the resource consumption of each service, such as memory and CPUs.
	Nginx、Apisix	API gateway.

Architecture

The core components of model inference microservices are as follows：

Type	Component	Description
Feature	feature engineering	user offline、user realtime、user seq features and item features.
Sample	recall/rank samples	create TFRcords format samples.
Recall	cf recall	user cf 、 item cf and swing.
	dssm recall	recall from dssm model and faiss index.
	simple recall	rules recall, such as hot items recall.
	cold start	new users and new items cold start.
Rank	pre_ranking	thousands items pre_ranking after recall .
	ranking	hundreds items ranking after pre_ranking.
	re_rank	hundreds items re_ranking after ranking .
Services	config loader	Sparse service's start config from Naocs, such as grpc info 、 redis info and index info.
	dubbo service	dubbo protocol service.
	gRPC service	grpc protocol service.
	rest service	restful service.
APIs	dubbo api	provide dubbo protocol api.
	gRPC api	provide gRPC protocol api.
	rest api	provide http protocol api.
Web	web	services manage and Service monitor page.
Deploy	faiss	faiss index service deploy.
	tfserving	tensorflow model deploy.
	infer	recommend system infer deploy.

Services Deploy

Docker
Kubernetes 
Nginx
Apisix
ELK

*扩展

1.推荐系统
王树森推荐系统公开课 - 基于小红书的场景讲解工业界真实的推荐系统。
● Recommender_System

2.YouTuBe推荐系统排序模型
以"DNN_for_YouTube_Recommendations"模型和电影评分数据集（ml-1m）为基础，详尽的展示了如何基于TensorFlow2实现推荐系统排序模型。
● YouTube深度排序模型(多值embedding、多目标学习)

3.机器学习 Sklearn入门教程
● 机器学习Sklearn入门教程

4.深度学习TensorFlow入门教程
● 深度学习TensorFlow入门教程