Reasoning about logged robot experiences

This page lists methods and tools for reasoning about log data of robot tasks created using the CRAM logging infrastructure. These logs consist of two parts: High-level plan events are stored as action instances in an OWL file, while lower-level high-volume data is logged separately into a MongoDB database. This allows efficient recording and storage of data like tf poses, perception results etc. Special Prolog predicates allow to access the information stored in the database.

Usually, the log files are created using the CRAM logging infrastructure that also includes the installation of a MongoDB database. If you would like to use the system on another computer, make sure to have a MongoDB database installed (the Ubuntu default version should work).

sudo apt-get install mongodb-dev

The logged data is stored in a database called 'roslog'; the collections in this database correspond to the respective topic names, for example 'tf'. From the KnowRob side, you will need the “Complete system including add-on packages” installation of KnowRob from source and compile mod_execution_trace using

rosmake mod_execution_trace

We have prepared an archive with test data logged during a pick-and-place task with the PR2 robot. You can download the file from here and insert it into your MongoDB using the following commands:

tar -xvjf pick-and-place.tar.bz2
cd pick-and-place
mongoimport --db roslog --collection tf tf.json
mongoimport --db roslog --collection logged_designators logged_designators.json
mongoimport --db roslog --collection logged_metadata logged_metadata.json

The knowrob_mongo package contains methods for accessing information in the MongoDB database via KnowRob predicates, for example the logged tf pose data or perception results.

$ rosrun rosprolog rosprolog knowrob_mongo
?- register_ros_package(mod_srdl).
?- owl_parse('owl/perceive-no-details.owl', false, false, true).

# read pose of cameras in /map coordinates at the beginning of event_dInF6Fvz
?- owl_individual_of(A, srdl2comp:'Camera'), 
   owl_has(A, srdl2comp:urdfName, literal(_Frame)), 
   atom_concat('/', _Frame, Frame), 
   owl_has('http://ias.cs.tum.edu/kb/execution-log-atvVerRS#event_dInF6Fvz', knowrob:startTime, Time), 
   mng_lookup_transform('/map', Frame, Time, Trans). 
   
   
# read latest designator that was logged before the end of event_dInF6Fvz
?- owl_has('http://ias.cs.tum.edu/kb/execution-log-atvVerRS#event_dInF6Fvz', knowrob:endTime, Time), 
   mng_latest_designator_before_time(Time, Type, Pose).

?- knowrob_mongo:obj_visible_in_camera('http://ias.cs.tum.edu/kb/PR2.owl#pr2_l_elbow_flex_link', 
'http://ias.cs.tum.edu/kb/PR2.owl#pr2_head_mount_kinect_rgb_link', 
'http://ias.cs.tum.edu/kb/namespace_LMutVndu#timepoint_1376577918').
true .

# Which camera can see the robot part A?
?- knowrob_mongo:obj_visible_in_camera('http://ias.cs.tum.edu/kb/PR2.owl#pr2_l_wrist_flex_link', 
Camera, 'http://ias.cs.tum.edu/kb/namespace_LMutVndu#timepoint_1376577918').
Camera = 'http://ias.cs.tum.edu/kb/PR2.owl#pr2_wide_stereo_l_stereo_camera_frame' ;
Camera = 'http://ias.cs.tum.edu/kb/PR2.owl#pr2_wide_stereo_r_stereo_camera_frame' ;
Camera = 'http://ias.cs.tum.edu/kb/PR2.owl#pr2_head_mount_kinect_ir_link' ;
Camera = 'http://ias.cs.tum.edu/kb/PR2.owl#pr2_head_mount_kinect_rgb_link' ;

The following is to be performed from the roslog_2013_08_29_1445 folder:

# Is some part of the right arm blocking the view on the object?
?- register_ros_package(mod_srdl), owl_parse('perceive-no-details.owl', false, false, true).
?- mng_obj_pose_by_desig('http://ias.cs.tum.edu/kb/cram_log.owl#VisualPerception_Z9fXhEae_object_0', Pose).
?- sub_component(pr2:pr2_right_arm, Part), 
   once(owl_individual_of(Part, srdl2comp:'UrdfLink')),
   obj_blocked_by_in_camera(log:'VisualPerception_Z9fXhEae_object_0', Part, 
   pr2:pr2_head_mount_kinect_rgb_link, log:'timepoint_1377780296').
   
# Is the object visible in the camera, given the camera's field of view?
?- obj_visible_in_camera(log:'VisualPerception_Z9fXhEae_object_0', 
                         pr2:pr2_head_mount_kinect_rgb_link,   
                         log:'timepoint_1377780219').
false.

?- obj_visible_in_camera(log:'VisualPerception_Z9fXhEae_object_0', 
                         pr2:pr2_head_mount_kinect_rgb_link,
                         log:'timepoint_1377780271').
true .

1) Get latest version of knowrob and knowrob_addons from Link 1 Link 2 respectively. Put these ROS stacks into your ROS workspace.

2) Get the dataset from Link (Note that dataset is slightly modified version of Jan's. Namely, some modifications were done in namespaces and class namings). Extract it somewhere you choose.

$ rosrun rosprolog rosprolog mod_vis
?- register_ros_package(mod_execution_trace).
?- owl_parse('path_of_dataset_should_be_here/perceive-no-details.owl', false, false, true).

“What is the probability of a certain task was failed during the execution of a plan?” is shown as an example. It is also one of the use-cases what we want to show in our ACS'13 paper.

?- aggregate_all(count,(task(Task), task_class(Task, 'http://ias.cs.tum.edu/kb/knowrob_cram.owl#AchieveGoalPerformOnProcessModule'),  failure_class(Error, Error_Class), failure_task(Error, Task)), Count), aggregate_all(count,(task(Task), task_class(Task, 'http://ias.cs.tum.edu/kb/knowrob_cram.owl#AchieveGoalPerformOnProcessModule')), Count2), Prob = Count/Count2.

Note that some predicates still don't work because I'm still implementing/modifying them

Please file issues here: https://github.com/knowrob/knowrob_addons/issues

Please file issues here: https://github.com/code-iai/iai_software/issues