Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
doc:marker_visualization [2014/06/05 11:38] – external edit 127.0.0.1doc:marker_visualization [2015/10/19 15:07] (current) – [Marker-based visualization canvas] daniel86
Line 1: Line 1:
 ====== Marker-based visualization canvas ====== ====== Marker-based visualization canvas ======
 +~~NOTOC~~
 +\\
 +^ This page describes the 'catkinized' version of KnowRob that uses the [[http://wiki.ros.org/catkin/|catkin buildsystem]] and the pure Java-based [[http://wiki.ros.org/rosjava|rosjava]]. The older version, which was based on the rosbuild buildsystem and rosjava_jni, used a [[http://wiki.ros.org/mod_vis|local visualization canvas implemented in Java]].^
 +\\
  
 {{ :doc:web-visualization.png?nolink&650 |}} {{ :doc:web-visualization.png?nolink&650 |}}
  
-Based on [[http://www.ros.org/wiki/rviz/DisplayTypes/Marker|rviz Markers]], this visualization module can display the current or past belief states in the rviz visualizer as well as in the ros3djs web-based visualization.+Based on [[http://www.ros.org/wiki/rviz/DisplayTypes/Marker|rviz Markers]], this visualization module can display 3D information in a browser-based canvas that has been built using the [[https://github.com/RobotWebTools/ros3djs|ros3djs library]].
  
-An example can be seen here (only from the local network): http://tenorth.ai.loc/~tenorth/test-vis.html+===== Marker Types =====
  
-**This module is currently under development and not publicly available.**+The set of supported marker primitive types can be queried using the ''marker_prop_type(Type,_)'' predicate. 
 +The primitive types include arrow, cube, sphere, cylinder, line_strip, line_list, cube_list, sphere_list, points, text_view_facing, mesh_resource and triangle_list. 
 +Primitive markers can be created using ''marker($type('Name'),_)'', where $type must be replaced by one of the primitive types.
  
 +In addition, KnowRob supports a set of complex marker types.
 +  * **cylinder_tf(Frame0,Frame1)**: A cylinder between 2 links
 +  * **link(Frame)**: A marker that takes the pose of a link
 +  * **trajectory(Frame)**: A trajectory of a link
 +  * **trail(Frame)**: A trail of a link (trajectory with alpha fade)
 +  * **average_trajectory(Frame)**: The average over multiple trajectories
 +  * **pointer(Frame0,Frame1)**: Arrow that points from Frame0 in the direction of Frame1
 +  * **object(Name)**: A marker that represents an OWL individual (including children objects)
 +  * **agent(Name)**: A marker that represents an OWL agent individual (including all links)
 +  * **stickman(Name)**: A special human skeleton visualization, kinematic structure is spanned by the OWL individual indentified by Name
 +===== Usage =====
  
-===== System overview ===== +First, setup the rosbridge communication interface and initialiaze the `knowrob_vis` package.
-{{ :doc:web-visualization-architecture.png?nolink |}}+
  
-**Note:** Though the visualization looks much like a standard web page, it's more similar to a web-based rviz. This means that there is still a single ROS graph and a single KnowRob instance in the background that is shared among all website visitors.+  $ roslaunch rosbridge_server rosbridge_websocket.launch 
 +  $ rosrun rosprolog  rosprolog knowrob_vis 
 +   
 +Markers must be explicitly published by the user with following query ([[http://www.open-ease.org/|openEASE]] does this automatically).
  
 +  % publish modified marker
 +  ?- marker_publish.
  
  
-===== Example queries ===== 
  
-The following queries read data from the semantic map and from the logged data and visualize it in the browser-based visualization. The robot pose at time 1377780296 is read from the MongoDB using the tfMemory extension.+==== Semantic Map Marker ==== 
 + 
 +The following queries read data from a semantic map and visualize it in the browser-based visualization.
  
-  $ roslaunch knowrob_vis knowrob_vis.launch 
-  $ rosrun rosprolog  rosprolog knowrob_vis 
-   
-  % load packages, start marker publisher 
-  ?- register_ros_package(knowrob_mongo),  
-     register_ros_package(mod_srdl),  
-     register_ros_package(mod_execution_trace),  
-     visualisation_canvas. 
-   
-  % load CRAM log file 
-  ?- owl_parse('pick-and-place-no-details.owl', false, false, true). 
-   
   % load and visualize semantic map   % load and visualize semantic map
-  ?- owl_individual_of(A, knowrob:'SemanticEnvironmentMap'), add_object_with_children(A). +  ?- owl_parse('package://knowrob_map_data/owl/ccrl2_semantic_map.owl'). 
-   +  ?- owl_individual_of(A, knowrob:'SemanticEnvironmentMap'), marker_update(object(A)).
-  % read start time of grasp action from MongoDB +
-  ?- task_goal(T, 'GRASP'), task_start(T, St).+
      
-  % read robot pose at this time and add to the visualization +  % highlight all electrical household appliances 
-  ?- mng_robot_pose_at_time(pr2:'PR2Robot1', '/map', $St, Pose), +  ?- owl_individual_of(A, knowrob:'ElectricalHouseholdAppliance'), marker_highlight(object(A)).
-     add_object_with_children(pr2:'PR2Robot1', $St). +
-   +
-  % highlight all cameras that are part of the SRDL model of the robot +
-  ?- owl_individual_of(A, srdl2comp:'Camera'), highlight_object(A, @(true), 0, 255, 255, 255).+
  
 +==== Agent Marker ====
  
 +The following queries read data from a robot description, look up the robot pose in the mongo DB
 +and visualize it in the browser-based visualization.
  
-The following is to be performed from the roslog_2013_08_29_1445 folder:+  % load and visualize the pr2 robot for given timepoint T 
 +  ?- owl_parse('package://knowrob_srdl/owl/PR2.owl'). 
 +  ?- mng_robot_pose_at_time(pr2:'PR2Robot1','/map',T,_), marker_update(object(pr2:'PR2Robot1'),End). 
 + 
 +==== Trajectory Marker ==== 
 + 
 +The following queries read data from a robot description, sample the '/base_link' between T0 and T1 
 +and visualize it in the browser-based visualization. 
 + 
 +  % load and visualize the base link of the pr2 robot for given timerange (T0,T1) 
 +  ?- owl_parse('package://knowrob_srdl/owl/PR2.owl'). 
 +  ?- marker_update(trajectory('/base_link'), interval(T0,T1,dt(Interval))). 
 +===== Visualizing 3D object meshes ===== 
 + 
 +KnowRob supports the visualization of objects using 3D surface meshes in the STL or Collada file formats. These meshes can either be attached to an object instance or to an object class, in which case they are used for visualization of all of its instances. The following OWL snippet shows how an instance can be linked to its CAD model using the //pathToCadModel// property: 
 + 
 +<code xml> 
 + <owl:NamedIndividual rdf:about="&map_obj;milk1"> 
 +     <rdf:type rdf:resource="&knowrob;CowsMilk-Product"/> 
 +     <knowrob:pathToCadModel rdf:datatype="&xsd;string"> 
 +                             package://knowrob_tutorial/cad/milk.kmz</knowrob:pathToCadModel> 
 + </owl:NamedIndividual> 
 +</code> 
 + 
 +The package //knowrob_cad_models// in //knowrob_addons// provides such definitions for several object types, and also contains CAD models that are downloaded at build time. 
 + 
 +Inside the ROS filesystem, resources such as binary files can be described using different kinds of URLs: The common file:%%//%% and http:%%//%% URLs, and the package:%%//%% URL used above that describes a path relative to a ROS package. These URLs can be resolved using the [[http://wiki.ros.org/resource_retriever|resource_retriever]] tool.  
 + 
 +Since the client of a webserver cannot access the whole file system, this mechanism does not work as such if requests are made from a website. The ros3djs library therefore rewrites package:%%//%% URLs for object meshes to subfolders of the website's docroot. For example, meshes in the package pr2_description are searched at http://localhost/pr2_description/... 
 + 
 +If you would like to visualize meshes contained in ROS packages in the browser, you have to put these meshes into the docroot of the webserver, i.e. as a subfolder of the directory 'html' in //knowrob_vis//. Please make sure that the folder is named exactly like the ROS package containing the meshes and that is has the same subfolders. You can remove all files except the meshes themselves.
  
-  # Is some part of the right arm blocking the view on the object? 
-  ?- register_ros_package(mod_srdl), owl_parse('perceive-no-details.owl', false, false, true). 
-  ?- mng_obj_pose_by_desig('http://ias.cs.tum.edu/kb/cram_log.owl#VisualPerception_Z9fXhEae_object_0', Pose). 
-  ?- sub_component(pr2:pr2_right_arm, Part),  
-     once(owl_individual_of(Part, srdl2comp:'UrdfLink')), 
-     obj_blocked_by_in_camera(log:'VisualPerception_Z9fXhEae_object_0', Part,  
-     pr2:pr2_head_mount_kinect_rgb_link, log:'timepoint_1377780296'). 
-      
-  # Is the object visible in the camera, given the camera's field of view? 
-  ?- obj_visible_in_camera(log:'VisualPerception_Z9fXhEae_object_0',  
-                           pr2:pr2_head_mount_kinect_rgb_link, 
-                           log:'timepoint_1377780271'). 
-  true . 
-   
  
 ===== Links ===== ===== Links =====