ros_intel_movidius_ncs

by intel

123 Stars 60 Forks Last release: over 2 years ago (v0.6.0) Apache License 2.0 91 Commits 4 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

rosintelmovidius_ncs

Warning: This repo is deprecated. For latest ROS wrapper for Movidius NCS, please refer to our project rosopenvinotoolkit.

See: https://github.com/intel/rosopenvinotoolkit!!

1 Introduction

The Movidius™ Neural Compute Stick (NCS) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. NCS is powered by the same low power high performance Movidius™ Vision Processing Unit (VPU) that can be found in millions of smart security cameras, gesture controlled drones, industrial machine vision equipment, and more.

This project is a ROS wrapper for NC API of NCSDK, providing the following features: * A ROS service for object classification and detection of a static image file * A ROS publisher for object classification and detection of a video stream from a RGB camera * Demo applications to show the capabilities of ROS service and publisher * Support multiple CNN models of Caffe and Tensorflow * Support multiple devices in parallel to acclerate inference

There are 2 active branches in this project: * master - stable branch
The latest version on it is v0.6.0 which supports NCSDK v1.12.00. master branch is only updated when every milestone release ready. * devel - default branch
This branch is updated from time to time and maintain the latest code on it. Each pull request should be submitted based on devel branch. We will merge patches to master branch on every milestone release.

2 Prerequisite

  • An x86_64 computer running Ubuntu 16.04
  • ROS Kinetic
  • Movidius Neural Compute Stick (NCS)
  • Movidius Neural Compute (MvNC) SDK
  • Movidius Neural Compute Application Zoo
  • RGB Camera, e.g. RealSense D400 Series or standard USB camera

3 Environment Setup

  • Install ROS Kinetic Desktop-Full (guide)
  • Create a catkin workspace (guide)
  • Install NCSDK v1.12.00 (github)
  • Install NC APP Zoo (github)
  • NCSDK should be installed in
    /opt/movidius
    by default. Create a symbol link in
    /opt/movidius
    to NC APP Zoo.
    Shell
    sudo ln -s /ncappzoo /opt/movidius/ncappzoo
    
    After that, make sure you can find graph data in
    /opt/movidius/ncappzoo/caffe
    or
    /opt/movidius/ncappzoo/tensorflow
    and image data in
    /opt/movidius/ncappzoo/data/images
  • Install ROS package for different cameras as needed. e.g.
    1. Standard USB camera
      Shell
      sudo apt-get install ros-kinetic-usb-cam
      
    2. RealSense D400 series camera
    3. Install Intel® RealSense™ SDK 2.0 (tag v2.9.1)
    4. Install from source code(Recommended)
    5. Install from package
         Note: Create a symbol link from libusb.a to libusb-1.0.a, otherwise "libusb.a" is probably not to be found by librealsense.
         
      sudo ln -s /usr/lib/x86_64-linux-gnu/libusb-1.0.a /usr/lib/libusb.a
    6. Install Intel® RealSense™ ROS (guide)
      Shell
      cd ~/catkin_ws/src
      git clone https://github.com/intel-ros/realsense.git
      cd  realsense
      git checkout 2.0.2
      cd ~/catkin_ws
      catkin_make
      
      ## 4 Building and Installation
      Shell
      # Building
      cd ~/catkin_ws/src
      git clone https://github.com/intel/object_msgs
      git clone https://github.com/intel/ros_intel_movidius_ncs.git
      cd ros_intel_movidius_ncs
      git checkout master
      cd ~/catkin_ws
      catkin_make
      # Installation
      catkin_make install
      source install/setup.bash
      # Copy label files from this project to the installation location of NCSDK
      cp ~/catkin_ws/src/ros_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/
      
      ## 5 Running the Demo ### 5.1 Classification #### 5.1.1 Supported CNN Models ###### Table1 |CNN Model|Framework|Usage| |:-|:-|:-| |AlexNet|Caffe|Image/Video| |GoogLeNet|Caffe|Image/Video| |SqueezeNet|Caffe|Image/Video| |Inceptionv1|Tensorflow|Image/Video| |Inceptionv2|Tensorflow|Image/Video| |Inceptionv3|Tensorflow|Image/Video| |Inceptionv4|Tensorflow|Image/Video| |MobileNet|Tensorflow|Image/Video| #### 5.1.2 Classification Result with GoogLeNet classification with googlenet #### 5.1.3 Running the Demo
  • Static Image
  • Video Streaming

5.2 Detection

5.1.1 Supported CNN Models

|CNN Model|Framework|Usage| |:-|:-|:-| |MobileNetSSD(Recommended)|Caffe|Image/Video| |TinyYolo_v1|Caffe|Image/Video|

5.1.2 Detection Result with MobileNetSSD

detection with mobilenetssd

5.1.3 Running the Demo

5.3 Inference with multiple NCS devices

We introduce multiple NCS support from v0.6.0, providing x1.8 / x2.5 / x3 performance improvement with 2NCSs / 3NCSs / 4NCSs compared with 1 NCS respectively. Refer here for more details.

6 Interfaces

6.1 Topic

Classification:

/movidius_ncs_nodelet/classified_objects

Detection:
/movidius_ncs_nodelet/detected_objects

6.2 Service

Classification:

/movidius_ncs_image/classify_object

Detection:
/movidius_ncs_image/detect_object

7 Known Issues

  • Only absolute path of image file supported in image inference demo
  • Only test on RealSense D400 series camera and Microsoft HD-300 USB camera
  • Current v0.6.0 supporting NCSDK v1.12.00 is on master branch. devel branch is the development branch for the next release.
  • Unit test for movidiusncslib failed due to one exception.

8 TODO

  • Support more CNN models
  • Support latest NCSDK
  • Support results display with Rviz
Any security issue should be reported using process at https://01.org/security

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.