Differences

This shows you the differences between two versions of the page.

--- doc:marker_visualization [2014/11/28 17:07] – admin
+++ doc:marker_visualization [2015/10/19 15:07] (current) – [Marker-based visualization canvas] daniel86
@@ Line 1: / Line 1: @@
 ====== Marker-based visualization canvas ======
+~~NOTOC~~
+\\
+^ This page describes the 'catkinized' version of KnowRob that uses the [[http://wiki.ros.org/catkin/|catkin buildsystem]] and the pure Java-based [[http://wiki.ros.org/rosjava|rosjava]]. The older version, which was based on the rosbuild buildsystem and rosjava_jni, used a [[http://wiki.ros.org/mod_vis|local visualization canvas implemented in Java]].^
+\\
 {{ :doc:web-visualization.png?nolink&650 |}}
-Based on [[http://www.ros.org/wiki/rviz/DisplayTypes/Marker|rviz Markers]], this visualization module can display the current or past belief states in the web-based visualization. The browser-based 3D canvas has been built using the [[https://github.com/RobotWebTools/ros3djs|ros3djs library]].
+Based on [[http://www.ros.org/wiki/rviz/DisplayTypes/Marker|rviz Markers]], this visualization module can display 3D information in a browser-based canvas that has been built using the [[https://github.com/RobotWebTools/ros3djs|ros3djs library]].
+===== Marker Types =====
-===== Visualizing object meshes =====
+The set of supported marker primitive types can be queried using the ''marker_prop_type(Type,_)'' predicate.
-Inside the ROS ecosystem, resources like binary files can be described using different kinds of URLs like file:%%//%%, http:%%//%% or package:%%//%%, which specifies the path relative to a ROS package. These URLs can be resolved using the [[http://wiki.ros.org/resource_retriever|resource_retriever]] tool. Since the webserver does not access to the whole file system, this mechanism does not work any more if requests are made from a website. The ros3djs library therefore rewrites package:%%//%% URLs to point to the docroot of the website. For example, meshes in the package pr2_description are now searched for at http://localhost/pr2_description.
+The primitive types include arrow, cube, sphere, cylinder, line_strip, line_list, cube_list, sphere_list, points, text_view_facing, mesh_resource and triangle_list.
+Primitive markers can be created using ''marker($type('Name'),_)'', where $type must be replaced by one of the primitive types.
-In order to visualize meshes contained in ROS packages, these meshes therefore have to be put into the docroot of the webserver, which is usually /var/www/<pkg_name>.
+In addition, KnowRob supports a set of complex marker types.
+  * **cylinder_tf(Frame0,Frame1)**: A cylinder between 2 links
+  * **link(Frame)**: A marker that takes the pose of a link
+  * **trajectory(Frame)**: A trajectory of a link
+  * **trail(Frame)**: A trail of a link (trajectory with alpha fade)
+  * **average_trajectory(Frame)**: The average over multiple trajectories
+  * **pointer(Frame0,Frame1)**: Arrow that points from Frame0 in the direction of Frame1
+  * **object(Name)**: A marker that represents an OWL individual (including children objects)
+  * **agent(Name)**: A marker that represents an OWL agent individual (including all links)
+  * **stickman(Name)**: A special human skeleton visualization, kinematic structure is spanned by the OWL individual indentified by Name
+===== Usage =====
-register_ros_package(knowrob_cad_models).
+First, setup the rosbridge communication interface and initialiaze the `knowrob_vis` package.
-Some of the objects in the demo set have CAD models attached. They are used for visualization as well as for reasoning (see below). These models are automatically used when they are specified for either a complete class of objects (using an OWL hasValue restriction) or for an object instance as shown below:
+  $ roslaunch rosbridge_server rosbridge_websocket.launch
-<code xml>
+  $ rosrun rosprolog  rosprolog knowrob_vis
- <owl:NamedIndividual rdf:about="&map_obj;milk1">
-     <rdf:type rdf:resource="&knowrob;CowsMilk-Product"/>
+Markers must be explicitly published by the user with following query ([[http://www.open-ease.org/|openEASE]] does this automatically).
-     <knowrob:widthOfObject rdf:datatype="&xsd;double">0.1</knowrob:widthOfObject>
-     <knowrob:depthOfObject rdf:datatype="&xsd;double">0.1</knowrob:depthOfObject>
-     <knowrob:heightOfObject rdf:datatype="&xsd;double">0.25</knowrob:heightOfObject>
-     <knowrob:pathToCadModel rdf:datatype="&xsd;string">
-                             package://knowrob_tutorial/cad/milk.kmz</knowrob:pathToCadModel>
- </owl:NamedIndividual>
-</code>
-The %%package://...%% identifier references files inside ROS packages.
+  % publish modified marker
+  ?- marker_publish.
+==== Semantic Map Marker ====
-===== Example queries =====
+The following queries read data from a semantic map and visualize it in the browser-based visualization.
-The following queries read data from the semantic map and from the logged data and visualize it in the browser-based visualization. The robot pose at time 1377780296 is read from the MongoDB using the tfMemory extension.
-  $ roslaunch knowrob_vis knowrob_vis.launch
-  $ rosrun rosprolog  rosprolog knowrob_vis
-  % load packages, start marker publisher
-  ?- register_ros_package(knowrob_mongo),
-     register_ros_package(mod_srdl),
-     register_ros_package(mod_execution_trace),
-     visualisation_canvas.
-  % load CRAM log file
-  ?- owl_parse('pick-and-place-no-details.owl', false, false, true).
   % load and visualize semantic map
-  ?- owl_individual_of(A, knowrob:'SemanticEnvironmentMap'), add_object_with_children(A).
+  ?- owl_parse('package://knowrob_map_data/owl/ccrl2_semantic_map.owl').
+  ?- owl_individual_of(A, knowrob:'SemanticEnvironmentMap'), marker_update(object(A)).
-  % read start time of grasp action from MongoDB
-  ?- task_goal(T, 'GRASP'), task_start(T, St).
-  % read robot pose at this time and add to the visualization
+  % highlight all electrical household appliances
-  ?- mng_robot_pose_at_time(pr2:'PR2Robot1', '/map', $St, Pose),
+  ?- owl_individual_of(A, knowrob:'ElectricalHouseholdAppliance'), marker_highlight(object(A)).
-     add_object_with_children(pr2:'PR2Robot1', $St).
-  % highlight all cameras that are part of the SRDL model of the robot
-  ?- owl_individual_of(A, srdl2comp:'Camera'), highlight_object(A, @(true), 0, 255, 255, 255).
+==== Agent Marker ====
+The following queries read data from a robot description, look up the robot pose in the mongo DB
+and visualize it in the browser-based visualization.
-The following is to be performed from the roslog_2013_08_29_1445 folder:
+  % load and visualize the pr2 robot for given timepoint T
+  ?- owl_parse('package://knowrob_srdl/owl/PR2.owl').
+  ?- mng_robot_pose_at_time(pr2:'PR2Robot1','/map',T,_), marker_update(object(pr2:'PR2Robot1'),End).
+==== Trajectory Marker ====
+The following queries read data from a robot description, sample the '/base_link' between T0 and T1
+and visualize it in the browser-based visualization.
+  % load and visualize the base link of the pr2 robot for given timerange (T0,T1)
+  ?- owl_parse('package://knowrob_srdl/owl/PR2.owl').
+  ?- marker_update(trajectory('/base_link'), interval(T0,T1,dt(Interval))).
+===== Visualizing 3D object meshes =====
+KnowRob supports the visualization of objects using 3D surface meshes in the STL or Collada file formats. These meshes can either be attached to an object instance or to an object class, in which case they are used for visualization of all of its instances. The following OWL snippet shows how an instance can be linked to its CAD model using the //pathToCadModel// property:
+<code xml>
+ <owl:NamedIndividual rdf:about="&map_obj;milk1">
+     <rdf:type rdf:resource="&knowrob;CowsMilk-Product"/>
+     <knowrob:pathToCadModel rdf:datatype="&xsd;string">
+                             package://knowrob_tutorial/cad/milk.kmz</knowrob:pathToCadModel>
+ </owl:NamedIndividual>
+</code>
+The package //knowrob_cad_models// in //knowrob_addons// provides such definitions for several object types, and also contains CAD models that are downloaded at build time.
+Inside the ROS filesystem, resources such as binary files can be described using different kinds of URLs: The common file:%%//%% and http:%%//%% URLs, and the package:%%//%% URL used above that describes a path relative to a ROS package. These URLs can be resolved using the [[http://wiki.ros.org/resource_retriever|resource_retriever]] tool.
+Since the client of a webserver cannot access the whole file system, this mechanism does not work as such if requests are made from a website. The ros3djs library therefore rewrites package:%%//%% URLs for object meshes to subfolders of the website's docroot. For example, meshes in the package pr2_description are searched at http://localhost/pr2_description/...
+If you would like to visualize meshes contained in ROS packages in the browser, you have to put these meshes into the docroot of the webserver, i.e. as a subfolder of the directory 'html' in //knowrob_vis//. Please make sure that the folder is named exactly like the ROS package containing the meshes and that is has the same subfolders. You can remove all files except the meshes themselves.
-  # Is some part of the right arm blocking the view on the object?
-  ?- register_ros_package(mod_srdl), owl_parse('perceive-no-details.owl', false, false, true).
-  ?- mng_obj_pose_by_desig('http://ias.cs.tum.edu/kb/cram_log.owl#VisualPerception_Z9fXhEae_object_0', Pose).
-  ?- sub_component(pr2:pr2_right_arm, Part),
-     once(owl_individual_of(Part, srdl2comp:'UrdfLink')),
-     obj_blocked_by_in_camera(log:'VisualPerception_Z9fXhEae_object_0', Part,
-     pr2:pr2_head_mount_kinect_rgb_link, log:'timepoint_1377780296').
-  # Is the object visible in the camera, given the camera's field of view?
-  ?- obj_visible_in_camera(log:'VisualPerception_Z9fXhEae_object_0',
-                           pr2:pr2_head_mount_kinect_rgb_link,
-                           log:'timepoint_1377780271').
-  true .
 ===== Links =====