Semantic map editor

The editor is designed as a simple graphical tool to facilitate the creation and debugging of semantic maps. It can load and export OWL descriptions of semantic environment maps and allows to create, delete, and modify the objects therein. Its main features are

• Easy creation of object instances
• Pose description using position vector, 4×4 pose matrix or quaternion
• Simple rotations around the different axes
• Keyboard shortcuts for the most important commands
• OWL import and export procedures

Semantic maps form a structured, semantic environment representation that consists of a collection of objects that are described as instances of object classes in an ontology. Each object instance inherits the properties of its class (e.g. the refrigerator inherits the property ‘storage place for perishable items’, or the oven the property ‘can be used for heating food’). In addition, instance-specific properties like the weight of an object can be specified.

There are two kinds of hierarchies related to the object representation: On the one hand, there is the specialization hierarchy of object classes that describes e.g. a Refrigerator as a specialization of ElectricalHouseholdAppliance as a specialization of ElectricalAppliance and Device. On the other hand, there is a component hierarchy (part-of hierarchy) that describes the composition of objects from parts, including the respective 6D poses of the components as well as articulated joints (prismatic and rotational).

CAD models (COLLADA files, PLY, etc) can be attached to an object class or instance to describe its geometry in more detail (and to get nicer visualizations).

The format in which the maps are described is the same OWL language that is also used for action recipes — though of course using different classes. This allows joint inference about actions and objects, for example to query for all objects that are part of a recipe in order to check if the robot has an object recognition model for each of them, or to check if the positions of all objects in the recipe are known, i.e. if there is an instance of the respective class.

The semantic map editor is part of the knowrob_gui repository that you can checkout from

git clone https://github.com/knowrob/knowrob_gui.git

The knowrob_gui directory needs to be in your catkin workspace. Please compile it using 'catkin_make' after checking out the repository. Knowrob_gui depends on the knowrob stack and its dependencies. Please have a look at the installation instructions if you haven't installed knowrob yet. You can then start the editor with the following command:

rosrun knowrob_gui KnowRobGUI

After starting the program, you can load a map from a local OWL file. If you click on “load map from file”, a file selection dialog will open in which you can select an OWL file to be opened. If you want to start with an empty map, just click on 'cancel'. If you have selected a map, it will be loaded and visualized.

The left part of the program visualizes the semantic map. Pieces of furniture are drawn in different shades of grey, doors in yellow, rotational hinges as blue cylinders (z-axis pointing along the cylinder towards the block marker), and prismatic joints as green cylinders. If the objects in the map have been linked to CAD models, these are shown instead of the generic block-based visualization.

Clicking on an object loads it into the form fields on the right side. There are three synchronized tabs for entering the object information in different ways: First, there is a simple form, allowing to set the position of the object and to rotate it around the axes by 90 degrees. Second, there is a matrix form to enter the 4×4 homography matrix that supports rotation, position, scaling, and shearing. Third, there is a position+quaternion input method to set the position and orientation. Data entered into either tab is being updated in the other ones when switching the tab.

The form fields have the following meaning:

• OBJECT ID: Name of the OWL identifier. Has to be unique and cannot be changed, except by deleting and re-creating an object
• NEW: Clear form fields to create a new object instance. Can also be done implicitly by using a new object ID, which automatically creates a new instance.
• DELETE: remove the currently selected object from the map
• ADD TO MAP: update the visualization with the information from the form fields, e.g. for adding a newly created object to the map (keyboard shortcut: RETURN)
• The grey list of object classes serves for selecting the type of the object. The current selection is shown in the head of the list.
• OBJECT DIMENSIONS: Dimensions of the object in meters.
• OBJECT POSITION: Position of the object center in global coordinates in meters. Analogous for the POSE MATRIX and POSE QUATERNION fields in the other tabs.
• ROT_X, ROT_Y, ROT_Z: Rotate the object around the x,y,z-axis by 90 degrees.
• SELECT PARENT button and PARENT text field: Maps can be hierarchically structured, i.e. the child objects of the map are pieces of furniture like cupboards, which can be further decomposed into doors, handles, etc. This part-of-relation can be set using the following steps:
  • First create the parent and the child
  • Select the child by clicking on it
  • Click on the SELECT PARENT button and click on the parent object
  • Alternatively: Enter the identifier of the parent into the PARENT form field
• EXPORT TO OWL: Enter a file name and click on the button to write the current state of the map to an OWL file.

Switching between the text fields can be done using TAB and SHIFT-TAB. For all textfields with numbers, the value can be increased and decreased using PAGE_UP and PAGE_DOWN when the field is selected. This can be used for easily moving object instances along the axes.

The interface was implemented using Processing and the controlP5 library.

The following steps describe how to create a map of three cupboards, one of which has a door with a hinge.

1. Start the editor, don't select a file but press 'cancel' instead
2. Click on the 'new' button, enter a unique name for the first object in the map (without URL/Namespace), e.g. cupboard1
3. Select the class 'Cupboard'
4. Enter the object dimensions, e.g. [0.5] [0.5] [0.8]. You can switch to the next form field using TAB.
5. Enter the object position, e.g. [0] [0] [0.4]
6. Hit ENTER or click on 'Add to map'
7. Save the map to an OWL file by typing in the file name and clicking on 'Export to OWL'. It is recommended to regularly save your work while creating the map by just clicking on this button from time to time.
8. To create the second and third object, just edit the object ID to 'cupboard2' and 'cupboard3' and change their positions, either by typing [0.5] and [1.0] in the form field X_POS or by hitting ENTER, clicking into the X_POS field and shifting the object with PG_UP
9. Create an instance 'door2' of 'Door' with the dimensions [0.5] [0.01] [0.8] at the position [0.5] [0.25] [0.4]
10. Click on 'cupboard2' and 'door2' to verify that both are loaded correctly.
11. Create a 'HingedJoint' called 'joint2' of arbitrary dimension at the same pose as the door.
12. Shift the hinge to the right position at the edge of the door using the PG_UP and PG_DOWN keys
13. Click on the JOINT PROPERTIES tab, click on 'select child' and click on the door in the map. Then click on 'select parent' and click on 'cupboard2'.
14. You can set the joint limits of the new hinge to whatever value you like

The resulting map should look like this: http://ias.cs.tum.edu/kb/demo-map.owl

The import and export from/to RoboEarth is deprecated since the RoboEarth project has ended and the availability of the server infrastructure is not guaranteed any more.

• Importing, editing and exporting a file may result in a loss of information that is contained in the original OWL file, but is not correctly parsed into the internal representation and therefore not exported any more. The layout of the OWL files will be modified, comments may be lost.
• List of object classes is currently hard-coded, but can be easily extended at the top of SemanticMapEditor.java

• Semantic Object Maps for Robotic Housework - Representation, Acquisition and Use (Dejan Pangercic, Moritz Tenorth, Benjamin Pitzer, Michael Beetz), In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.
• KNOWROB-MAP — Knowledge-Linked Semantic Object Maps (Moritz Tenorth, Lars Kunze, Dominik Jain, Michael Beetz), In 10th IEEE-RAS International Conference on Humanoid Robots, 2010.