Object Analytics (OA) is ROS wrapper for realtime object detection, localization and tracking. These packages aim to provide real-time object analyses over RGB-D camera inputs, enabling ROS developer to easily create amazing robotics advanced features, like intelligent collision avoidance and semantic SLAM. It consumes sensor_msgs::PointClould2 data delivered by RGB-D camera, publishing topics on object detection, object tracking, and object localization in 3D camera coordination system.
OA keeps integrating with various "state-of-the-art" algorithms. * Object detection offload to GPU, rosopenclcaffe, with Yolo v2 model and OpenCL Caffe framework * Object detection offload to VPU, rosintelmovidius_ncs (devel branch), with MobileNet SSD model and Caffe framework
ROS packages from ros-kinetic-desktop-full * roscpp * nodelet * stdmsgs * sensormsgs * geometrymsgs * dynamicreconfigure * pclconversions * cvbridge * libpcl-all * libpcl-all-dev * ros-kinetic-opencv3
Other ROS packages * object_msgs * rosintelmovidius_ncs or opencl_caffe
NOTE: OA depends on tracking feature from OpenCV (3.3 preferred, 3.2 minimum). The tracking feature is recently provided by ROS Kinetic package "ros-kinetic-opencv3" (where OpenCV 3.3.1 is integrated). However, if you're using an old version of ROS Kinetic (where OpenCV 3.2 is integrated), tracking feature is not provided. In such case you need self-build tracking from opencv_contrib. It is important to keep opencv_contrib (self-built) and opencv (ROS Kinetic provided) in the same OpenCV version that can be checked from "/opt/ros/kinetic/share/opencv3/package.xml"
bash cd ${ros_ws} # "ros_ws" is the catkin workspace root directory where this project is placed in catkin_make
to test
bash catkin_make run_tests
to install
bash catkin_make install
RGB-D camera * librealsense2 tag v2.9.1 and realsenseroscamera tag 2.0.2 if run with Intel RealSense D400
roslaunch realsense_ros_camera rs_rgbd.launch* openni_launch or freenect_launch and their dependencies if run with Microsoft XBOX 360 Kinect
bash roslaunch openni_launch openni.launch* rosastracamera if run with Astra Camera
bash roslaunch astra_launch astra.launch
bash roslaunch object_analytics_launch analytics_opencl_caffe.launch
bash roslaunch object_analytics_launch analytics_movidius_ncs.launch
Frequently used options * input_points Specify arg "inputpoints" for the name of the topic publishing the sensor_msgs::PointCloud2 messages by RGB-D camera. Default is "/camera/depthregistered/points" (topic compliant with ROS OpenNI launch) * aging_th Specifiy tracking aging threshold, number of frames since last detection to deactivate the tracking. Default is 16. * probability_th Specify the probability threshold for tracking object. Default is "0.5".
bash roslaunch object_analytics_launch analytics_movidius_ncs.launch aging_th:=30 probability_th:="0.3"
object_analytics/rgb (sensor_msgs::Image)
object_analytics/pointcloud (sensor_msgs::PointCloud2)
object_analytics/localization (objectanalyticsmsgs::ObjectsInBoxes3D)
object_analytics/tracking (objectanalyticsmsgs::TrackedObjects)
object_analytics/detection (object_msgs::ObjectsInBoxes)
topic | fps | latency sec | |
OpenCL Caffe | |||
localization | 6.63 | 0.23 | |
detection | 8.88 | 0.17 | |
tracking | 12.15 | 0.33 | |
Movidius NCS | |||
localization | 7.44 | 0.21 | |
detection | 10.5 | 0.15 | |
tracking | 13.85 | 0.24 |
Steps to enable visualization on RViz are as following
bash roslaunch object_analytics_visualization rviz.launch