Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
doc:docker:dev [2014/08/27 07:17] admindoc:docker:dev [2015/03/31 12:51] (current) – [Sharing images via Docker hub] daniel86
Line 1: Line 1:
-====== KnowRob & Docker --- developer documentation ======+====== KnowRob & Docker -- developer documentation ======
  
-===== Building the images from Dockerfiles =====+This page lists information for developers that would like to modify, update and develop the docker images. If you would only like to start the existing docker containers, please have a look [[/doc/docker|here]].
  
  
-Dockerfiles can be found in this repository: https://github.com/knowrob/docker+===== Building Docker images=====
  
-You can build docker images from them using +The Dockerfiles for the KnowRob containers can be found in this repository: https://github.com/knowrob/docker. You can build docker images from them using 
-  cd ~/docker/knowrob-daemon +<code bash> 
-  docker build -t knowrob/hydro-knowrob-daemon . +cd ~/docker/hydro-knowrob/hydro-knowrob-base 
-  cd ~/docker/knowrob-interactive +docker build -t knowrob/hydro-knowrob-base . 
-  docker build -t knowrob/hydro-knowrob-interactive . + 
-  cd ~/docker/webapp +cd ~/docker/hydro-knowrob/hydro-knowrob-daemon 
-  docker build -t knowrob/webrob .+docker build -t knowrob/hydro-knowrob-daemon . 
 + 
 +cd ~/docker/hydro-knowrob/hydro-knowrob-interactive 
 +docker build -t knowrob/hydro-knowrob-interactive . 
 + 
 +cd ~/docker/webapps/easeapp 
 +docker build -t openease/easeapp . 
 + 
 +cd ~/docker/webapps/login 
 +docker build -t openease/login . 
 + 
 +cd ~/docker/webapps/knowrob 
 +docker build -t openease/knowrob . 
 +</code>
  
 You can tag a version of an image using You can tag a version of an image using
-  docker tag 18df3323a0d8 knowrob/webrob:0.1.0 +  docker tag 18df3323a0d8 openease/knowrob:0.1.0
-===== Running KnowRob and the Web application =====+
  
 +===== Sharing images via Docker hub =====
 +Instead of building all images from Dockerfiles, you can also upload and download them from/to Docker hub. The push/pull interaction is very similar to git and uses the 'tags' associated with an image for describing which one to share:
 +
 +<code bash>
 +  # upload image:
 +  docker push knowrob/hydro-knowrob-daemon
 +
 +  # download images (this can also be done with the 'update-containers' script)
 +  docker pull knowrob/hydro-knowrob-daemon
 +  docker pull knowrob/knowrob_data
 +  docker pull knowrob/user_data
 +  docker pull openease/easeapp
 +  [...]
 +</code>
 +
 +
 +===== Running Docker containers =====
 +
 +==== KnowRob containers ====
 +
 +There are two versions of the KnowRob container: The interactive one starts a Bash shell that can be used for console-based interaction with the system. The other one is a daemonized version that starts KnowRob and json_prolog as a daemon without interactive shell.
 +<code bash>
   # interactive:   # interactive:
   docker run -t -P -i knowrob/hydro-knowrob-interactive /bin/bash   docker run -t -P -i knowrob/hydro-knowrob-interactive /bin/bash
Line 23: Line 57:
   # daemon mode:   # daemon mode:
   docker run -t -P -d knowrob/hydro-knowrob-daemon   docker run -t -P -d knowrob/hydro-knowrob-daemon
-  +</code>
  
-===== Managing data ===== 
  
-The approach is to use [[https://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container|data-only containers]]+==== MongoDB ==== 
 +<code bash> 
 +  docker run -d -v /data/db --name mongo_data busybox true 
 +  docker run --p 27017:27017 --volumes-from mongo_data --name mongo_db mongo 
 +</code>
  
-There is one container per user (called <user>_data), one for the common parts (called knowrob_data). The user container will serve as sandbox in which users can store their files, the knowrob_data container will provide commonly used data sets. 
  
-The common data container knowrob_data is [[https://docs.docker.com/userguide/dockerrepos/#automated-buildsauto-built]] on [[https://hub.docker.com/|DockerHub]] from the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GithHub repository.+=== Importing data into MongoDB ===
  
 +If you want to import data into a dockerized MongoDB instance, first start an interactive container that mounts the host directory which contains the exported json files:
 +<code bash>
 +  docker run -t -P --rm --link mongo_db:mongo \
 +         -v /home/tenorth/work/roslog:/var/roslog \
 +         -i knowrob/hydro-knowrob-interactive /bin/bash
 +</code>
 +Then you can run the 'mongoimport' program from within the container. Make sure to set the host correctly, i.e. to refer to the MongoDB instance in the other container:
 +<code bash>
 +  cd /var/roslog/<experiment-id>
 +  mongoimport --host "$MONGO_PORT_27017_TCP_ADDR:$MONGO_PORT_27017_TCP_PORT" --db roslog --collection tf tf.json
 +</code>
 +
 +
 +
 +===== Managing data using containers =====
 +
 +Docker containers are cheap and usually short-lived, i.e. they can easily be started, stopped, removed and updated. The problem is how to store persistent data such as the content of a database or user-specific files. The approach we use are [[https://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container|data-only containers]] -- normal containers that contain nothing but a data volume that can be mounted by the other containers that provide the applications. These data-only containers shall not be removed, but can persist over longer time, also if the application containers get updated.
 +
 +In the KnowRob use case, there is one container for user data (called user_data), and another one for common data (called knowrob_data). The user container will serve as sandbox in which users can store their files, the knowrob_data container will provide commonly used data sets. The common data container knowrob_data is [[https://docs.docker.com/userguide/dockerrepos/#automated-builds|auto-built]] on [[https://hub.docker.com/|DockerHub]] from the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GithHub repository.
  
-==== Creating the user data containers ==== 
-The user containers are built from the user_data Dockerfile. It basically just creates a user 'ros' in order to set the permissions correctly and creates the directory '/home/ros/sandbox'. The following commands build a Docker image from this Dockerfile (needs to be done only when the Dockerfile gets updated) and start a container, in this case for the user 'demo'. 
-  cd docker/user-sandbox 
-  docker build -t knowrob/user_data . 
-  docker run --name demo_data knowrob/user_data true 
  
 ==== Updating the common data container ==== ==== Updating the common data container ====
 The knowrob_data image is automatically built whenever new data is pushed to the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GitHub repository. This image is, however, not automatically updated in the server's registry. In order to pull the newest version and to replace the running container with a new one, use the following commands: The knowrob_data image is automatically built whenever new data is pushed to the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GitHub repository. This image is, however, not automatically updated in the server's registry. In order to pull the newest version and to replace the running container with a new one, use the following commands:
  
 +<code bash>
   docker pull knowrob/knowrob_data:latest    docker pull knowrob/knowrob_data:latest 
   docker rm knowrob_data   docker rm knowrob_data
   docker run --name knowrob_data knowrob/knowrob_data:latest true   docker run --name knowrob_data knowrob/knowrob_data:latest true
 +</code>
 +
  
 ==== Starting a user container that includes the data volumes ==== ==== Starting a user container that includes the data volumes ====
-If the KnowRob image has been built as described earlier, it can be started using the following command such that both the common and the user-specific data volumes are mounted: 
  
 +If the KnowRob image has been built as described earlier, it can be started using the following command such that both the common and the user-specific data volumes are mounted:
 +<code bash>
   # interactive mode   # interactive mode
-  docker run -t -P -i \+  docker run -t -P -i --rm \
              --volumes-from knowrob_data \              --volumes-from knowrob_data \
-             --volumes-from demo_data \+             --volumes-from user_data \
              --name demo \              --name demo \
 +             --link mongo_db:mongo \
              knowrob/hydro-knowrob-interactive \              knowrob/hydro-knowrob-interactive \
              /bin/bash              /bin/bash
      
   # daemon mode   # daemon mode
-  docker run -t -P -d \+  docker run -t -P -d --rm\
              --volumes-from knowrob_data \              --volumes-from knowrob_data \
-             --volumes-from demo_data \+             --volumes-from user_data \
              --name demo \              --name demo \
 +             --link mongo_db:mongo \
              knowrob/hydro-knowrob-daemon              knowrob/hydro-knowrob-daemon
-   +  </code> 
-These examples are for the user 'demo', so 'demo' and 'demo_data' have to be replaced with the respective username. +These examples are for the user 'demo', so 'demo' has to be replaced with the respective username.
- +
-==== Starting MongoDB ==== +
- +
-  docker run -d -v /data/db --name mongo_data busybox true +
-  docker run -d -p 27017:27017 --volumes-from mongo_data --name mongo_db mongo +
- +
-==== Importing data into a MongoDB ==== +
- +
-If you want to import data into a dockerized MongoDB instance, first start an interactive container that mounts the host directory which contains the exported json files: +
-  docker run -t -P --rm --link mongo_db:mongo \ +
-         -v /home/tenorth/work/roslog:/var/roslog \ +
-         -i knowrob/hydro-knowrob-interactive:1.0.2 /bin/bash +
- +
-Then you can run the 'mongoimport' program from within the container. Make sure to set the host correctly, i.e. to refer to the MongoDB instance in the other container: +
-  mongoimport --host "$MONGO_PORT_27017_TCP_ADDR:$MONGO_PORT_27017_TCP_PORT" --db roslog --collection tf tf.json +
-   +
-==== Starting the web app in a container ==== +
- +
-The web app needs access to the Docker socket, so you have to start it like this: +
-  docker run --rm -i -p 5000:5000 -v /var/run/docker.sock:/var/run/docker.sock knowrob/webrob python webrob.py +
- +
-You can then access the Web interface at http://localhost:5000 +
- +
- +
-==== Pulling the containers manually ==== +
- +
-  docker pull knowrob/hydro-knowrob-daemon +
-  docker pull knowrob/webrob +
-  docker pull knowrob/knowrob_data +
-  docker pull knowrob/user_data+