DevOpsCICD_final

OVERALL FLOWCHART

CREATE AZURE RESOURCES

Create Azure Resources

Microsoft custom templates- Create Resource Group, Iot Hub and Container Registry and device provisioning service.

Create the Iot Edge device identify after you have created the hub using Azure CLI.

az iot hub device-identity create --hub-name IoTEdgeDevOps-iothub-qxw --device-id sp3 --edge-enabled

Create a VM for Iot Edge runtime using the following command. Use the following Azure CLI command to create your IoT Edge device based on the prebuilt iotedge-vm-deploy template

az deployment group create \
--resource-group IoTEdgeResources \
--template-uri "https://aka.ms/iotedge-vm-deploy" \
--parameters dnsLabelPrefix='<REPLACE_WITH_VM_NAME>' \
--parameters adminUsername='azureuser' \
--parameters deviceConnectionString=$(az iot hub device-identity connection-string show --device-id myEdgeDevice --hub-name <REPLACE_WITH_HUB_NAME> -o tsv) \
--parameters authenticationType='password' \
--parameters adminPasswordOrKey="<REPLACE_WITH_PASSWORD>"

Example:

 az deployment group create \
--resource-group  IoTEdgeResources \
--template-uri "https://aka.ms/iotedge-vm-deploy" \
--parameters dnsLabelPrefix='myiot678' \
--parameters adminUsername='azureuser' \
--parameters deviceConnectionString=$(az iot hub device-identity connection-string show --device-id sp3 --hub-name IoTEdgeDevOps-iothub-qxw -o tsv) \
--parameters authenticationType='password' \
--parameters adminPasswordOrKey="<REPLACE_WITH_PASSWORD>"

Verify that you have the following resources in your resource group.

Run the following commands to test the status of the IoT Edge device:

sudo systemctl status iotedge

CREATE YOUR OBJECT DETECTION MODEL USING MICROSOFT COGNITIVE SERVICES

Login to the Custom Vision AI Portal
Train your model.

Export your model.

Choose export as Docker File and export as linux container.

IOT EDGE SOLUTION — CUSTOM MODULES AND COMPILATION

Download Visual Studio Code.
Install Azure Iot Tools extension in Visual studio.
Create an IoT Edge solution, with a custom python module. Watch this [https://www.youtube.com/watch?v=PrbpsnHU3CQ&t=2s] Video for more details.

CAMERA MODULE

First, we create a module that read images from camera, and send to our classifier to perform inference. Here we are just going to have a static picture.
Check github for camera specific code.
Make sure you replace the container registry details in the module.json file.

OBJECT DETECTION MODULE

Add a new Iot Solution using Visual Studio Code.
Unzip the file we downloaded from Custom Vision, and copy to the folder contains module above. There are 4 files copied, as shown below:

If you expand “app”, you will see a python script to perform prediction, as well as the model exported from custom vision. If you notice, there are several Dockerfile over here. Tell IoT Edge to use “Dockerfile” (The one without other extension) during compilation by modifying “module.json”.
Make sure you replace the container registry details in the module.json file.

{
 "$schema-version": "0.0.1",
 "description": "",
 "image": {
   "repository": "iotedgedevopsqxw.azurecr.io/medium_classifier",
   "tag": {
     "version": "0.0.1",
     "platforms": {
       "amd64": "./Dockerfile",
       "amd64.debug": "./Dockerfile"
     }
   },
   "buildOptions": [],
   "contextPath": "./"
 },
 "language": "python"
}

Now we will import our repo in the DevOps and build a CI/CD pipeline.

PUSH ALL CODE TO GITHUB

Push your code in your visual studio workspace to Github.

DevOps — Repo and Pipeline

CI Pipline

Navigate to DevOps
Create a new Organisation and create a new project inside it with visibility private.

Go to Repos and Files and click on import a new repository.

Enter the link to your github repo.

Go to pipelines and press use the classic editor.

Select Azure repos Git and press continue.

Select your repo, and start with “Empty job” on the template page. This is the place for us to configure automation steps via the agent hosted in cloud.

Select Agent specification as Ubuntu-18.04 or 16.04.

In the tasks section, Click on *“+” to add tasks, navigate to Marketplace, and search for “Azure IoT Edge”

Add the module. Under “Action”, select “Build module images”. Browse your repo to indicate the “deployment.template.json” as well. Leave the rest default

Add a new task using the “Azure IoT Edge” extension. After building images, we need to push to our Azure Container Registry. Now, the action change to “Push module images”

Copy files to our artifact repository for deployment. Add a task called “Copy files”. Select your repository as Source Folder, enter the following 2 lines for contents that we need to copy, and “$(Build.ArtifactStagingDirectory)” as our target folder.

Finally, we need to publish artifacts. Add a new task using “Publish build artifacts”. Ensure the path to publish is same as above, and put “drop” as Artifact name.

Enable Continous integration

Go to variables and enter your container registery details (CONTAINER_REGISTRY_USERNAME,CONTAINER_REGISTRY_PASSWORD,CONTAINER_REGISTRY_ADDRESS) in devops.

These can be found on portal using by pressing access keys in the container registry.

Once done, click Save & Queue and proceed with the compilation. Your pipeline should build as follows showing all completed steps.

This completed the CI pipeline.

CD Pipeline

Navigate to “Releases” on left hand side and click “New release pipeline”. Similarly, we will start with empty job under template page.
Start with Artifacts. Click “Add an artifact”, select the right source, and change “source alias” to “drop”. Recalled in previous step, we called our artifacts “drop”.

Enable continous deployment by pressing the trigger button.

Click on “0 task” to add tasks. The steps here is similar to above.

Add task to generate deployment manifest for the Iot Edge device.

Set output path to: “$(System.DefaultWorkingDirectory)/drop/drop/configs/deployment.json”.
Set .template.json.file to $(System.DefaultWorkingDirectory)/drop/drop/deployment.template.json

Add a new task using the “Azure IoT Edge” extension. Set Action to deploy to the Iot Edge device. Select your IoT Hub, and device ID that you wish to push modules to.

Add the variables in the release pipleine as well as we did for the CI pipeline.

Back to the repo, we need to update the code for “deployment.template.json”. Under registryCredentials, update to the following:

"username": "${CONTAINER_REGISTRY_USERNAME}",
"password": "${CONTAINER_REGISTRY_PASSWORD}",
"address": "${CONTAINER_REGISTRY_ADDRESS}"

For environmental variable of camera modules, we can indicate the container create options in this file as well. Navigate to the modules, under the camera modules, replace with the following. Remember to replace with your module’s name.

"createOptions": "{\"HostConfig\":{\"Privileged\":true},\"Env\":[\"videosource=/dev/video0\",\"classifierapi=http://<classifier module name>:80/image\"]}"