Skip to content
This repository has been archived by the owner on Jan 7, 2023. It is now read-only.

Commit

Permalink
Upgrade to v0.5.0
Browse files Browse the repository at this point in the history 
* Update README.md

Current v0.3.0 supporting NCSDK v1.07.06 is on master branch. devel branch is the development branch for the next release.

* find the catkin package before calling catkin_package() function in cmake file. Without this, the catkin tools from python-catkin-tools(not catkin_make) will report build issue. (#2)

* use x86 intrinsic to optimize the float conversion

* use x86 intrinsic to optimize the float conversion

* fix the missing conversion for intrinsics usage

* Support NCSDKv1.09.00

* Support NCSDK v1.09.00

1. change the launch file
2. update APIs to replace deprecated APIs
3. statistic information is passed by argument instead of txt file

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

support NCSDK v1.09.00

* Update README.md

* Support realsense d400 series camera

* support realsense d400 series camera

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* Update README.md

* change the resolution of rs_d400 and install target to install folder

* support realsense d400 series camera

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* change default w/h and install target to install folder

width=640, height=480

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Skip inferring when no subscriber

Signed-off-by: Peter Han <peter.han@intel.com>

* Init member variables of NcsImpl with proper value

Signed-off-by: Peter Han <peter.han@intel.com>

* Quick fix a build issue


* quick fix a build issue

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* change version in package.xml to V0.4.0

* support realsense d400 series camera

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* change default w/h and install target to install folder

width=640, height=480

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* change the version in package.xml to v0.4.0

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* Add a changelog file

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update CHANGELOG.rst

* add a changelog file

* Update CHANGELOG.rst

* enable tiny yolo on both ncs_image and ncs_stream

* support realsense d400 series camera

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* enable tiny yolo both on ncs_image and ncs_stream

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update ncs_usbcam_detector.launch

* change the default graph path in launch files

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* Fix compile issue on Intel old platforms when using x86 intrinsic optimization

Add "mf16c" compiling flag for platform earlier than Ivy Bridge.

Signed-off-by: Chao Li <chao1.li@intel.com>

* format code according to ros coding guide

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* refactor code (#23)

* refactor code

1. redesign the classes to make transaction logic more clear.
2. remove the duplicated attributes in different classes.
3. optimize code structure
4. further change coding style according to ROS and google coding guide

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* change permission of files and folders

Signed-off-by: Huang,Xiaojun <xiaojun.huang@intel.com>

* change the format of category files (#24)

change the format of category files from dos to unix to avoid unrecognized chars

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* remove zr300 launch file

stop supporting realsense zr300 camera due to its EOL.

* remove deprecated launch files

* remove deprecated launch files

Use yaml files to configure different CNN models, which can greatly reduce the redundency of launch files.
Besides, use if clause to judge different kinds of cameras in launch files also can reduce the number of launch files.

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* Show exception messages (#27)

* enable mobilenet_ssd support

* enable mobilenet_ssd support

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update alexnet.yaml

* Update README.md

* change the mobilenet_ssd path to the one in ncappzoo

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* support inception_v1

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* support inception_v2

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* support inception_v3

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* fix the issue of "undefined reference to opencv"

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* support inception_v4

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* support mobilenet

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* fix the incorrect inference result from mobilenetssd

ignore the locaton values which are lower than 0 or higher than image range

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* add test images

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Reorganize readme file

* seperate classification and detection description

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update README.md

* Change title of description files with more meaningful names (#39)

* change the title of description files with more meaningful names

Signed-off-by: Xiaojun Huang <xiaojun.huang@intel.com>

* Update image_classification.md

* minor change readme file and other description files for v0.5.0 release

* Update image_classification.md

* Update video_classification.md

* Update image_detection.md

* Update video_detection.md

* Update README.md
  • Loading branch information
@xhuan28
xhuan28 authored Dec 21, 2017
1 parent 2c520e7 commit 0fce023
Show file tree
Hide file tree
Showing 68 changed files with 3,847 additions and 1,865 deletions.
218 changes: 57 additions & 161 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
The Movidius™ Neural Compute Stick ([NCS](https://developer.movidius.com/)) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. NCS is powered by the same low power high performance Movidius™ Vision Processing Unit ([VPU](https://www.movidius.com/solutions/vision-processing-unit)) that can be found in millions of smart security cameras, gesture controlled drones, industrial machine vision equipment, and more.

This project is a ROS wrapper for NC API of [NCSDK](https://movidius.github.io/ncsdk/), providing the following features:
* A ROS service for classifying objects in a static image file
* A ROS publisher for classifying objects in a video stream from a RGB camera
* A ROS service for object classification and detection of a static image file
* A ROS publisher for object classification and detection of a video stream from a RGB camera
* Demo applications to show the capabilities of ROS service and publisher
* Support multiple [CNN models](#table2)
* Support multiple CNN models of Caffe and Tensorflow

There are 2 active branches in this project:
* master - *stable branch*
The latest version on it is v0.4.0 which supports NCSDK v1.10.00. master branch is only updated when every milestone release ready.
The latest version on it is v0.5.0 which supports NCSDK v1.11.00. master branch is only updated when every milestone release ready.
* devel - *default branch*
This branch is updated from time to time and maintain the latest code on it. Each pull request should be submitted based on devel branch. We will merge patches to master branch on every milestone release.

Expand All @@ -20,26 +20,25 @@ There are 2 active branches in this project:
* ROS Kinetic
* Movidius Neural Compute Stick (NCS)
* Movidius Neural Compute (MvNC) SDK
* RGB Camera, e.g. RealSense ZR300, RealSense D400 Series or standard USB camera
* Movidius Neural Compute Application Zoo
* RGB Camera, e.g. RealSense D400 Series or standard USB camera

## 3 Environment Setup
* Install ROS Kinetic Desktop-Full ([guide](http://wiki.ros.org/kinetic/Installation/Ubuntu))
* Create a catkin workspace ([guide](http://wiki.ros.org/catkin/Tutorials/create_a_workspace))
* Install NCSDK [v1.10.00](https://github.com/movidius/ncsdk/releases) ([github](https://github.com/movidius/ncsdk))
* NCSDK should be installed in ```/opt/movidius``` by default. Create a symbol link in ```/opt/movidius``` to the supported CNN network from NCSDK release package.
* Install NCSDK [v1.11.00](https://github.com/movidius/ncsdk/releases) ([github](https://github.com/movidius/ncsdk))
* Install NC APP Zoo([github](https://github.com/movidius/ncappzoo))
* NCSDK should be installed in ```/opt/movidius``` by default. Create a symbol link in ```/opt/movidius``` to NC APP Zoo.
```Shell
sudo ln -s <your-workspace>/ncsdk/examples /opt/movidius/examples
sudo ln -s <your-workspace>/ncappzoo /opt/movidius/ncappzoo
```
After that, make sure you can find graph data in ```/opt/movidius/examples/caffe``` or ```/opt/movidius/examples/tensorflow``` and image data in ```/opt/movidius/examples/data/images```
After that, make sure you can find graph data in ```/opt/movidius/ncappzoo/caffe``` or ```/opt/movidius/ncappzoo/tensorflow``` and image data in ```/opt/movidius/ncappzoo/data/images```
* Install ROS package for different cameras as needed. e.g.
1. Standard USB camera
```Shell
sudo apt-get install ros-kinetic-usb-cam
```
2. RealSense ZR300 camera
- Install RealSense camera package ([guide](https://github.com/IntelRealSense/realsense_samples_ros)). Refer to [Realsense ROS WiKi](http://wiki.ros.org/RealSense) for more information.

3. RealSense D400 series camera
```
2. RealSense D400 series camera
- Install Intel® RealSense™ SDK 2.0 ([guide](https://github.com/IntelRealSense/librealsense/blob/master/doc/distribution_linux.md)). Refer [here](https://github.com/IntelRealSense/librealsense) for more details about Intel® RealSense™ SDK 2.0.
- Install Intel® RealSense™ ROS ([guide](https://github.com/intel-ros/realsense))
```Shell
Expand All @@ -52,170 +51,67 @@ After that, make sure you can find graph data in ```/opt/movidius/examples/caffe
```
## 4 Building and Installation
```Shell
# Building
cd ~/catkin_ws/src
git clone https://github.com/intel/ros_intel_movidius_ncs.git
cd ros_intel_movidius_ncs
git checkout v0.4.0
git checkout devel
cd ~/catkin_ws
catkin_make
# Installation
catkin_make install
source install/setup.bash
# Copy label files from this project to the installation location of NCSDK
cp ~/catkin_ws/src/ros_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/
```
Copy ```categories.txt``` from this project to NCSDK installation location.
```Shell
cp ~/catkin_ws/src/ros_intel_movidius_ncs/data/categories.txt /opt/movidius/examples/data/ilsvrc12/
```

## 5 Running the Demo
### 5.1 Video Streaming Inference
#### 5.1.1 NCS and Standard USB Camera
Launch video streaming nodelet. Refer [here](#table1) for more parameter configurations.
```Shell
roslaunch movidius_ncs_launch ncs_usbcam.launch
```
Make sure you can get result from the topic of object classification.
```Shell
rostopic echo /movidius_ncs_nodelet/classified_object
```
Launch image viewer to show the inference result.
```Shell
roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/usb_cam/image_raw"
```
#### 5.1.2 NCS and RealSense ZR300 Camera
Launch video streaming nodelet. Refer [here](#table1) for more parameter configurations.
```Shell
roslaunch movidius_ncs_launch ncs_realsense_zr300.launch
```
Make sure you can get result from the topic of object classification.
```Shell
rostopic echo /movidius_ncs_nodelet/classified_object
```
Launch image viewer to show the inference result. You can launch it directly without setting ```camera_topic```, as its default value is ```/camera/color/image_raw```
```Shell
roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/camera/color/image_raw"
#or
roslaunch movidius_ncs_launch ncs_stream_example.launch
```
#### 5.1.3 NCS and RealSense D400 Series Camera
Launch video streaming nodelet. Refer [here](#table1) for more parameter configurations.
```Shell
roslaunch movidius_ncs_launch ncs_realsense_d400.launch
```
Make sure you can get result from the topic of object classification.
```Shell
rostopic echo /movidius_ncs_nodelet/classified_object
```
Launch image viewer to show the inference result. You can launch it directly without setting ```camera_topic```, as its default value is ```/camera/color/image_raw```
```Shell
roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/camera/color/image_raw"
#or
roslaunch movidius_ncs_launch ncs_stream_example.launch
```
#### 5.1.4 NCS and Other ROS Supported Camera
Launch your preferred camera node.
```Shell
#launch ROS master
roscore
#launch camera node in another console
rosrun <your-camera-pkg> <your-camera-node>
```
e.g.
```Shell
#launch ROS master
roscore
#launch camera node in another console
rosrun usb_cam usb_cam_node
```
Launch NCS nodelet and assign ```input_topic``` with the topic url of your rgb camera.
```Shell
roslaunch movidius_ncs_launch ncs_nocam.launch input_topic:=<your_rgb_camera_topic>
```
e.g.
```Shell
roslaunch movidius_ncs_launch ncs_nocam.launch input_topic:="/usb_cam/image_raw"
```
Launch image viewer to show the inference result.
```Shell
roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:=<your_rgb_camera_topic>
```
e.g.
```Shell
roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/usb_cam/image_raw"
```
### 5.2 Static Image Inference
Launch object classification service. Refer [here](#table1) for more parameter configurations.
```Shell
roslaunch movidius_ncs_launch ncs_image.launch
```
Run the example application with an absolute path of an image
```Shell
rosrun movidius_ncs_example movidius_ncs_example_image <image_path_to_be_inferred>
```
e.g.
```Shell
rosrun movidius_ncs_example movidius_ncs_example_image /opt/movidius/examples/data/images/cat.jpg
```
### 5.3 Choose Different CNN Models
You can choose different CNN Models for inference through the argument of ```graph_file_path```. e.g.
Using RealSense camera
```Shell
#one console
roslaunch movidius_ncs_launch ncs_realsense_d400.launch graph_file_path:="/opt/movidius/examples/caffe/SqueezeNet/graph"
#another console
roslaunch movidius_ncs_launch ncs_stream_example.launch
```
Using standard USB camera
```Shell
#one console
roslaunch movidius_ncs_launch ncs_usbcam.launch graph_file_path:="/opt/movidius/examples/caffe/AlexNet/graph"
#another console
roslaunch movidius_ncs_launch ncs_stream_example.launch camera_topic:="/usb_cam/image_raw"
```
### 5.1 Classification
#### 5.1.1 Supported CNN Models
###### *Table1*
|CNN Model|Framework|Usage|
|:-|:-|:-|
|AlexNet|Caffe|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#alexnet)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#alexnet)|
|GoogleNet|Caffe|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#googlenet)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#googlenet)|
|SqueezeNet|Caffe|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#squeezenet)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#squeezenet)|
|Inception_v1|Tensorflow|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#inception_v1)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#inception_v1)|
|Inception_v2|Tensorflow|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#inception_v2)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#inception_v2)|
|Inception_v3|Tensorflow|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#inception_v3)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#inception_v3)|
|Inception_v4|Tensorflow|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#inception_v4)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#inception_v4)|
|MobileNet|Tensorflow|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md#mobilenet)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md#mobilenet)|
#### 5.1.2 Classification Result with GoogleNet
![classification with googlenet](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/data/results/googlenet_dog.png "classification with googlenet")
#### 5.1.3 Running the Demo
* [Static Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_classification.md)
* [Video Streaming](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_classification.md)
### 5.2 Detection
#### 5.1.1 Supported CNN Models
|CNN Model|Framework|Usage|
|:-|:-|:-|
|MobileNetSSD|Caffe|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_detection.md#mobilenet_ssd)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_detection.md#mobilenet_ssd)|
|TinyYolo_v1|Caffe|[Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_detection.md#tinyyolo_v1)/[Video](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_detection.md#tinyyolo_v1)|
#### 5.1.2 Detection Result with MobileNetSSD
![detection with mobilenetssd](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/data/results/mobilenetssd_car_bicycle.png "detection with mobilenetssd")
#### 5.1.3 Running the Demo
* [Static Image](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/image_detection.md)
* [Video Streaming](https://github.com/intel/ros_intel_movidius_ncs/blob/devel/doc/video_detection.md)

## 6 Interfaces and Arguments
## 6 Interfaces
### 6.1 Topic
/movidius_ncs_nodelet/classified_object
Classification: ```/movidius_ncs_nodelet/classified_objects```
Detection: ```/movidius_ncs_nodelet/detected_objects```
### 6.2 Service
/movidius_ncs_image/classify_object
### 6.3 Arguments
###### *Table1*
|Node|Arguments|Default Value|Description|
|:-|:-|:-|:-|
|ncs|input_topic|/camera/color/image_raw|subscribed rgb camera topic|
|ncs|output_topic|/movidius_ncs_nodelet/classified_object|published topic of inference results|
|ncs|device_index|0|ncs device index|
|ncs|log_level|1|ncs log level|
|ncs|graph_file_path|/opt/movidius/examples/caffe/GoogLeNet/graph|the path of CNN model graph file|
|ncs|category_file_path|/opt/movidius/examples/data/ilsvrc12/categories.txt|object category list|
|ncs|network_dimension|224|network dimension for input data|
|ncs|channel1_mean|0.40787054|mean value of the first channel of image, defualt value is for ImageNet dataset|
|ncs|channel2_mean|0.45752458|mean value of the second channel of image, defualt value is for ImageNet dataset|
|ncs|channel3_mean|0.48109378|mean value of the third channel of image, defualt value is for ImageNet dataset|
|ncs|top_n|3|the number of results to be shown|
|realsense|color_width|640|frame width|
|realsense|color_height|480|frame height|
|usb cam|image_width|640|frame width|
|usb cam|image_height|480|frame height|
|usb cam|video_device|/dev/video0|use camera device node|
## 7 CNN Support Status
###### *Table2*
|CNN Model|Weights|Status|
|:-|:-|:-|
|GoogleNet|[weights](http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel)|Supported|
|AlexNet|[weights](http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel)|Supported|
|SqueezeNet|[weights](https://github.com/DeepScale/SqueezeNet/raw/master/SqueezeNet_v1.0/squeezenet_v1.0.caffemodel)|Supported|
|Gender|[weights](https://dl.dropboxusercontent.com/u/38822310/gender_net.caffemodel)|Supported|
|Age|[weights](https://dl.dropboxusercontent.com/u/38822310/age_net.caffemodel)|Not supported|
Classification: ```/movidius_ncs_image/classify_object```
Detection: ```/movidius_ncs_image/detect_object```

## 8 Known Issues
## 7 Known Issues
* Only absolute path of image file supported in image inference demo
* Only test on RealSense ZR300 camera, RealSense D400 series camera and Microsoft HD-300 USB camera
* Current v0.4.0 supporting NCSDK v1.10.00 is on master branch. devel branch is the development branch for the next release.
* Only test on RealSense D400 series camera and Microsoft HD-300 USB camera
* Current v0.5.0 supporting NCSDK v1.11.00 is on master branch. devel branch is the development branch for the next release.

## 9 TODO
## 8 TODO
* Support multiple NCS devices
* Support more CNN models
* Support latest NCSDK


###### *For security issues, please send mail to xiaojun.huang@intel.com*
###### *Any security issue should be reported using process at https://01.org/security*
Binary file added data/images/bicycle.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/images/bus.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/images/car.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/images/car_bicycle.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/images/car_bus.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/images/cat.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/images/dog.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added data/images/dog_bicycle.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 0fce023

Please sign in to comment.