Deliverables

• Basic: CUDA/OpenCV detects hand movement up/down
• Extra: Implement detection of additional gestures

Prerequisites

1. CUDA must be installed (See Lab 4).

2. Install OpenCV (instructions copied from the jetson wiki).

   1. Download the "OpenCV4Tegra for L4T 21.x" from here.

   2. Run the following in a terminal

      sudo dpkg -i libopencv4tegra-repo_l4t-r21_2.4.10.1_armhf.deb
      sudo apt-get update
      sudo apt-get install libopencv4tegra libopencv4tegra-dev
      

3. Get the OpenCV samples as part of the full OpenCV source code.

4. Test that everything is working right.

Requirement

Based off of a OpenCV sample, implement the detection of hand movements. Alternatively, you can detect another object of your choice as long as it is reasonable by size and the detection works properly. You may or may not decide to futher optimize the given code, or detect motions beyond up and down.

Provided Source

The bgfg example from class is here with Makefile.

This diff might be useful to see what needed to be changed to port the CPU code to the GPU.

Discussion

Camera setup

Connect the PlayStation eye to the Jetson TK1 board via the USB port. Normally, the kernel has to be rebuilt in order to be able to use the PS eye on the jetson tk1. We used the grinch kernel that already includes the necessary device drivers, so we are all set! You can open up the GUI window showing what is being capture by the webcam by using the following command:

cheese

You should be able to confirm that the camera works by the video shown on the screen.

Hand Detection

The hand detection code uses OpenCV and the lab jetsons have OpenCV installed on them so you should have no problem compiling and getting the code to work. Study the given code carefully as you should understand the algorithm in order to make appropriate changes. Try changing the code so that it would detect an object other than the hand, or to consistently detect your hand in the first place. You might find the following site to be helpful:

Helpful link

Good luck!

Notes

Compile example with:

g++ bgfg_segm.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_video -lopencv_gpu

Compile CPU version with:

g++ bgfg_segm.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_video

• NVIDIA OpenCV to GPU porting
• OpenCV GPU example
• CPU OpenCV Hand Detection
  • Amy's excellent walkthrough

Completed Lab Reports

• Tyler Smallwood
• Bryan Matsumoto
• Andy Russell
• Patrick Lu
• Sebastian Krupa
• Henry Huang
• Andrew Fishberg

This lab was developed by Da Eun Shim.