Object information pipeline

The recognition of objects may be handled by several nodes. There is a dataflow of object information between those nodes, which is explained in the following.

Node 1. Object information service

The "Object information service" can be any node providing a service to request the most recently known information of objects. This can be a database, or a Gazebo plugin as the one provided in the package gazebo_state_plugins (also described on this wiki page of gazebo-pkgs).

This node provides a service of type object_msgs/ObjectInfo.srv (to be found in repository general-message-pkgs) which can be used to retrieve object information.

Node 2. MoveIt! collision object generator

The "MoveIt! collision object generator" in package moveit_object_handling subscribes to messages of type object_msgs/Object (to be found in repository general-message-pkgs), converts them to the right message type accepted by MoveIt!, and then forwards that information to MoveIt! every time such a object_msgs/Object message arrives.

The collision object generator is a node which constantly runs in the background and is launched with moveit_object_handling/launch/collision_object_generator.launch.

The parameters of this ROS node can be set in:

rosed moveit_object_handling CollisionObjectsGenerator.yaml

• Make sure that moveit_get_planning_scene_topic and collision_object_topic are set to the right topic names of MoveIt!, but the defaults should normally work.
• Important is the parameter request_object_service which has to be the same as in the object information service (see Node 1. above).
• Also pay attention to the world_objects_topic which will need to be the same for the object detection (see Node 3. below).
• use_planning_scene_diff usually should be true as this works better, but you may also try to use false instead.

Note
The collision object generator only queries the object information in cases in which the geometry has to be re-aquired because the MoveIt! node lost information about the object (e.g. the MoveIt! node is restarted, or the object had been attached to the robot, and has been detached again, so geometry information is lost). This is necessary when only the first Object.msg message contains the geometry, and follow-up messages only contain the current pose of the object. This is why it is labeled "[Fallback]" an the graphic above. Otherwise, the collision object generator is independent of the object information. The dependency is only linked to a service of type object_msgs/ObjectInfo.srv, so the "object information" can be exchanged with any implementation providing this service.

Node 3. Object recognition

The "object recognition" represents any object recognizer implementation. For each object recognized, the node is expected to send out an object_msgs/Object message which can be received by the MoveIt! collision object generator (Node 2 above).

Object recognition can happen continuously (Object.msg messages sent repeatedly) or only at random times when the object is recongized.

Note: Given that the object is expected to move (e.g. after it has been picked up by the robot), it is better to have messages repeatedly being sent, so that MoveIt! can consider it in future motion plans.

An example of an object recognition implementing this interface can be found in the gazebo-pkgs "Gazebo fake object recogniser".

Why so many nodes?

The justification for this set-up is as follows:

• The object recognition should plug in with any other algorithm, in place of the object recognition (Node 3 above). Hence, the object recognition should be its own package which can be replaced individually.
• There will always be a need for a separate MoveIt! collision object generator (Node 2 above) which specifically sends on any object information to MoveIt!. This should be a node which can work with any way the object information is obtained - be it with a fake object recognition as described in the gazebo-pgks wiki, or a real object recognition. It should also not depend on any specific implementation such as the Gazebo object information service. Hence, it must be its own package.
• The object information service (Node 1 above) is specifically to obtain messages in the standard object_msgs/Object format. When using Gazebo, this is the only node which is a Gazebo plugin which directly queries Gazebo. There should not have to be other nodes which act as a Gazebo plugin.
• The general implementation of object_msgs was decided upon at the time to avoid dependencies of any of the packages to a specific message package, like for example object_recognition_msgs. While it would be good to use a common message type like object_recognition_msgs/RecognizedObject instead, it will require a bit more work to make it compatible with moveit_msgs/CollisionObject, which has the "primitives" field. Aside from that, specific services (like ObjectInfo.srv) have to be added as a message package anyway. A future migration away from object_msgs to object_recognition_msgs is possible, but it will require a bit more work.

Additional info: The package gazebo_state_plugins which provides an object information service for Gazebo, also offers the functionality to publish all gazebo objects such that they are received by the MoveIt! collision object generator directly, without the "detour" around the fake object recognition. But we want more control, in that we want it to use only the objects which we have explicitly set as "recognized", and only for the duration as they have been set to be "recognized". This more closely mimmicks the behaviour of a real object recognition system.

Tutorial

Please also refer to this tutorial for an example how to test the object information pipeline with the example jaco hand in jaco-arm-pkgs.