Computer Vision Starter Pack

Cheap Nvidia Computer Vision Camera

Overview

I have recently been toying around with computer vision more. I was looking for a low cost Nvidia GPU and camera sensor to bootstrap some projects. Sure you can started programming inside a Google Colab notebook, but I am more interested in embedded systems and applied machine learning.

The goal is to start programming real-time camera pipelines, converting state of the art models into TensorRT, and using Nvidia optimized computer vision libraries for things like optical flow and slam. My go to strategy in hardware is always to start small. I want to fully utilize the computing resources before leveling up.

I quick look at prices and you can see that you actually pay more per cuda core as you go up the product line. The Jetson Nano and TX2 both at around $1 per core. A closer look at the specs, shows the Jetson Xavier series boards also include tensor cores for doing 8 bit operations, and a dedicated vision processing module which may justify the higher price.

So, I decided to build out on a standard Jetson Nano Development kit. Even though the board is listed at $128 for 1k volume, I was able to find one Amazon for a little over $100. My go to strategy in hardware is always to start small. I want to fully utilize the computing resources before leveling up. This guide will go over how I put it all together.

Materials

Procedure

This is a pretty standard build for Nvidia newcomers like myself. I followed resources available from the Nvidia developer's documentation/forums and Jetson Hacks helpful guides.

0. Flash and bootup Jetson Nano

Before we get started, make sure you have ordered and received your Jetson Nano Developer Kit, and Samsung Micro SD card. Hopefully you have also have a spare monitor with an HDMI port laying around.

The first step is to flash your SD card with Nvidia's Jetpack OS, which is essentially Ubuntu with some extra software and Nvida SDKs that come pre-installed. Plug in the MicroSD card into your day to day computer for the initial flashing.

Follow the instructions from the Jetson Nano Get Started guide to write the image to your SD card. The guide includes references for mac, linux, and windows. Plugin the flashed SD card into the back of the device, and provide power through the Micro USB port.

$ lsb_release -a # Ubuntu Version

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

$ cat /etc/nv_tegra_release # Jetpack Version

# R32 (release), REVISION: 4.2, GCID: 20074772, BOARD: t210ref, 
# EABI: aarch64, DATE: Thu Apr  9 01:22:12 UTC 2020

$ nvcc --version # Cuda Version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_21:14:42_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

1. Internet Connection and SSH

The developer board comes with an ethernet port. Simply hook it up to your router. Keep in mind you may need to relocate your router next to the nano device or get a really long ethernet cable.

Run this shell script on your nano device to get the LAN IP address. Then we can connect over ssh to the device. ssh USERNAME@LANIP

NETCONF=$(ifconfig)
INETS=$(echo "${NETCONF}" | grep "inet ")
IPADDRS=$(echo "${INETS}" | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b")
LANIP=$(echo "${IPADDRS}" | grep 192 | grep -v 255)
echo "${LANIP}"

It usually much more comfortable to develop over SSH with your day to day machine. If you need to visualize a real-time display then you will need to use the nano board directly.

2. Attach/Configure Sony IMX219 Camera Sensor

Pretty simple to install the camera into one of the two CSI ports on the Jetson Nano, make sure the contact stripts are facing towards the device and secure the clip for stable contact.

Now you can check if the camera is working by capturing a live video stream and displaying the video on a monitor.

gst-launch-1.0 nvarguscamerasrc sensor_id=0 ! nvoverlaysink

Gstreamer is a great tool for handling video feeds. Highly recommend the intro tutorials which use the C API. Your Jetpack Gstreamer comes with hardware acceleration out of the box.

nvarguscamerasrc is a Nvidia tool for automating camera bring up and configurations. It uses the libargus API under the hood. You can further configure camera parameters like whitebalance, and exposure using this higher level api.

You may notice some pink vignnetting on the outside edges of the frame. We can easily download some ISP overrides to solve this issue. This is a common issue with wide angle lenses.

wget https://www.waveshare.com/w/upload/e/eb/Camera_overrides.tar.gz
tar zxvf Camera_overrides.tar.gz 
mv camera_overrides.isp /var/nvidia/nvcam/settings/
sudo chmod 664 /var/nvidia/nvcam/settings/camera_overrides.isp
sudo chown root:root /var/nvidia/nvcam/settings/camera_overrides.isp

3. Fan and Power Supply Setup

At this point you are up and running with your nano device. If you try executing some of the DeepStream example projects which use neural nets on video streams you will start to see your nano heat up. You can run tegrastats to diagnose your nano.

$ tegrastats

RAM 3029/3956MB (lfb 46x4MB) SWAP 429/1978MB (cached 15MB) 
CPU [24%@921,23%@921,19%@921,15%@921] 
EMC_FREQ 0% GR3D_FREQ 51% PLL@4aa 
CPU@47a PMIC@100C GPU@46.5C AO@53C 
thermal@46.75C POM_5V_IN 5716/5591 
POM_5V_GPU 2204/2170 POM_5V_CPU 708/845

Now it is time to install a proper power supply and fan. The power supply provides 4A * 5V = 20W of power, 10W more than you were getting with the micro usb setup. In order to use barrel jack instead of the microusb, place the jumper pin provided in the development to cover J48 Power Jack/USB Power Select Jumper. Plug power supply into barrel and the Nano should now boot up.

To attach the fan, first use an M3 threading tool to expand the holes on top of the heatsink. I used an Aligle 5pcs Brand New Daily Household Steel Silver Adjustable T Type One-Hand Ratchet Wrench with M3-M8 Taps Threading Tool (M3-M8) to get the job done. Next use M3 25mm hex screws to bolt the fan to the heatsink.

4. Enclosure and Camera Mount

Attach power and reset button to the correct pions and connect the CSI camera before screwing the board into the base of the box. Geekworm has a nice video that goes over pin locations.

I inserted the camera mount panel to the opposite of the CSI ports to allow the ribbon cable to lay flight when packed. I did not install the reset button on my build in case I accidentally press it.

Conclusion

Ok, we now have a fully functioning and powerful computer vision camera to work with. The whole build costs far less than $200. In the next experiment, we will build a realtime camera that runs object detection on the gpu. This seems like the go to newb computer vision project, and at the time of writing this, I am a newb.

timxor / jetson-nano-box