This repository provides a reference architecture of Edge Video Services (EVS). It includes instructions to build and deploy a video analytics service tailored for operator's infrastructure using Kubernetes in the Azure public MEC and Azure IoT Edge, and video ML containers like NVIDIA Triton Inference Server, Microsoft Rocket, and Intel OpenVINO Model Server. The repository also contains a sample video to count vehicles entering a parking lot for end-to-end testing.
Figure 1 shows the architecture of a video analytics pipeline deployed in an operator's infrastructure. The two-layer edge hierarchy consists of an on-premise (on-prem) edge (an Azure IoT Edge device), and a network edge (an Azure Kubernetes Service, or AKS on Azure public MEC).
- On-prem edge: An edge machine which sits on enterprise premise. It could be a customer-owned edge box or a compute cluster consisting of multiple work stations.
- Network edge: A beefy edge (e.g., Azure public MEC) with optimized connectivity to the on-prem edge and the cloud. Typically, it consists of a cluster of machines with accelerators like GPU.
Figure 1: A sample two-layer edge hierarchy with one on-prem edge node and one network edge cluster.
EVS provides the platform to ingest and orchestrate a cascaded video pipeline for analytics by splitting the pipeline across the edges. An EVS Client container is running on each edge to supply frames to various video ML containers. This way, EVS partitions the video processing workloads to make the best use of the available compute and network resources in the edge and cloud infrastructure.
❗Note: Video ML containers supported by EVS include Microsoft Rocket, Intel OpenVINO Model Server, and NVIDIA Triton Inference Server which already have support to plug in popular models for object detection and classification.
Other types of edge hierarchies: While the example above uses a two-layer edge hierarchy, EVS is generally applicable to all types of edge hierarchies. For instance, a three-layer edge hierarchy could have a camera edge (e.g., a smart camera) sitting together with an on-prem edge (e.g., a Kubernetes cluster on bare metal Linux) in a private 5G network before accessing the public MEC (Figure 2).
Figure 2: A three-layer edge hierarchy with a camera edge, an on-prem edge (cluster), and a network edge.
Deployment Instructions: In what follows, we use two-layer hierarchy (Figure 1) as an example, and describe the steps to deploy EVS on an on-prem edge (mimicked by an Azure VM as an IoT Edge Device) and a network edge (an AKS cluster in the public MEC). In the two-layer hierarchy, a CPU-based Rocket container is deployed to the on-prem edge for motion-based object detection. When an objects cross lines of interest in the frame, frames will be encoded and sent to the AKS cluster in the network edge for DNN-based object detection using NVIDIA Triton inference container with COCO pretrained YOLOv3. As a result, EVS reduces network traffic between the edges and provides efficient compute usage in public MEC.
We also provide a sample mp4 video along with a line configuration file so that you can run the pipeline end-to-end with an example to count vehicles entering a parking lot. You will also be able to test the deployment on your own videos (both pre-recorded ones and live RTSP streams). To learn more about configuring lines of interest, please visit this page.
Throughout the samples, we refer to two different terms. Here are descriptions for each of them for future reference:
- Development PC: the machine you use to clone the repo, run Jupyter Notebooks, connect to AKS cluster (e.g., via Azure and Kubernetes CLI), and remote login to any of the compute devices and the on-prem Kubernetes control node. We use a Windows 10 desktop in this example.
- Compute Device: any of the machines run EVS and video analytics containers. That includes the Azure IoT Edge device, and nodes in network edge clusters.
You may use your existing Development System (e.g., your desktop or laptop) or refer below link to create and setup a new Azure Virtual machine.
-
(1.1) Create development PC - Azure virtual machine (if you are using your existing Development System, you may skip this step and move to 1.2)
-
(1.2) Configure development PC
-
(3.2) Create Azure services
Now let's walk through the steps for deploying and orchestrating a video pipeline across the two-layer edge hierarchy (i.e., IoT Edge Device + AKS in Figure 1).
- (7.1) Read and follow the section to learn how to monitor and interpret the output.
- (7.2) Read and follow the section to learn how to reset the IoT edge device.
| Setup | On-premise Edge | Network Edge Cluster | Testing Status |
|---|---|---|---|
| 1 | Azure VM - OS: Ubuntu 20.04 | Azure Kubernetes Service - VM Size: Standard_NC6s_v2 | Passed |
• Azure public MEC
• Microsoft and AT&T demonstrate 5G-powered video analytics
• Video analytics at the edge, an ideal technology for 5G cloud monetization
• Telstra creates innovative AI solutions for the 5G era with Azure Video Analyzer
• Azure for Operators
| Folders | Description |
|---|---|
| common | collection of deployment instructions and Jupyter notebooks for environment setup |
| templates | template deployment manifest for on-prem and network edge |
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.