Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Last revisionBoth sides next revision
doc:docker:dev [2014/08/27 07:17] – admindoc:docker:dev [2015/03/31 12:50] – [Building Docker images] daniel86
Line 1: Line 1:
-====== KnowRob & Docker --- developer documentation ======+====== KnowRob & Docker -- developer documentation ======
  
-===== Building the images from Dockerfiles =====+This page lists information for developers that would like to modify, update and develop the docker images. If you would only like to start the existing docker containers, please have a look [[/doc/docker|here]].
  
  
-Dockerfiles can be found in this repository: https://github.com/knowrob/docker+===== Building Docker images=====
  
-You can build docker images from them using +The Dockerfiles for the KnowRob containers can be found in this repository: https://github.com/knowrob/docker. You can build docker images from them using 
-  cd ~/docker/knowrob-daemon +<code bash> 
-  docker build -t knowrob/hydro-knowrob-daemon . +cd ~/docker/hydro-knowrob/hydro-knowrob-base 
-  cd ~/docker/knowrob-interactive +docker build -t knowrob/hydro-knowrob-base . 
-  docker build -t knowrob/hydro-knowrob-interactive . + 
-  cd ~/docker/webapp +cd ~/docker/hydro-knowrob/hydro-knowrob-daemon 
-  docker build -t knowrob/webrob .+docker build -t knowrob/hydro-knowrob-daemon . 
 + 
 +cd ~/docker/hydro-knowrob/hydro-knowrob-interactive 
 +docker build -t knowrob/hydro-knowrob-interactive . 
 + 
 +cd ~/docker/webapps/easeapp 
 +docker build -t openease/easeapp . 
 + 
 +cd ~/docker/webapps/login 
 +docker build -t openease/login . 
 + 
 +cd ~/docker/webapps/knowrob 
 +docker build -t openease/knowrob . 
 +</code>
  
 You can tag a version of an image using You can tag a version of an image using
-  docker tag 18df3323a0d8 knowrob/webrob:0.1.0 +  docker tag 18df3323a0d8 openease/knowrob:0.1.0
-===== Running KnowRob and the Web application =====+
  
 +===== Sharing images via Docker hub =====
 +Instead of building all images from Dockerfiles, you can also upload and download them from/to Docker hub. The push/pull interaction is very similar to git and uses the 'tags' associated with an image for describing which one to share:
 +
 +<code bash>
 +  # upload image:
 +  docker push knowrob/hydro-knowrob-daemon
 +
 +  # download images (this can also be done with the 'update-containers' script)
 +  docker pull knowrob/hydro-knowrob-daemon
 +  docker pull knowrob/webrob
 +  docker pull knowrob/knowrob_data
 +  docker pull knowrob/user_data
 +  [...]
 +</code>
 +
 +
 +===== Running Docker containers =====
 +
 +==== KnowRob containers ====
 +
 +There are two versions of the KnowRob container: The interactive one starts a Bash shell that can be used for console-based interaction with the system. The other one is a daemonized version that starts KnowRob and json_prolog as a daemon without interactive shell.
 +<code bash>
   # interactive:   # interactive:
   docker run -t -P -i knowrob/hydro-knowrob-interactive /bin/bash   docker run -t -P -i knowrob/hydro-knowrob-interactive /bin/bash
Line 23: Line 57:
   # daemon mode:   # daemon mode:
   docker run -t -P -d knowrob/hydro-knowrob-daemon   docker run -t -P -d knowrob/hydro-knowrob-daemon
-  +</code>
  
-===== Managing data ===== 
  
-The approach is to use [[https://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container|data-only containers]]+==== MongoDB ==== 
 +<code bash> 
 +  docker run -d -v /data/db --name mongo_data busybox true 
 +  docker run -d -p 27017:27017 --volumes-from mongo_data --name mongo_db mongo 
 +</code>
  
-There is one container per user (called <user>_data), one for the common parts (called knowrob_data). The user container will serve as sandbox in which users can store their files, the knowrob_data container will provide commonly used data sets. 
  
-The common data container knowrob_data is [[https://docs.docker.com/userguide/dockerrepos/#automated-buildsauto-built]] on [[https://hub.docker.com/|DockerHub]] from the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GithHub repository.+=== Importing data into MongoDB ===
  
 +If you want to import data into a dockerized MongoDB instance, first start an interactive container that mounts the host directory which contains the exported json files:
 +<code bash>
 +  docker run -t -P --rm --link mongo_db:mongo \
 +         -v /home/tenorth/work/roslog:/var/roslog \
 +         -i knowrob/hydro-knowrob-interactive /bin/bash
 +</code>
 +Then you can run the 'mongoimport' program from within the container. Make sure to set the host correctly, i.e. to refer to the MongoDB instance in the other container:
 +<code bash>
 +  cd /var/roslog/<experiment-id>
 +  mongoimport --host "$MONGO_PORT_27017_TCP_ADDR:$MONGO_PORT_27017_TCP_PORT" --db roslog --collection tf tf.json
 +</code>
 +
 +
 +
 +===== Managing data using containers =====
 +
 +Docker containers are cheap and usually short-lived, i.e. they can easily be started, stopped, removed and updated. The problem is how to store persistent data such as the content of a database or user-specific files. The approach we use are [[https://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container|data-only containers]] -- normal containers that contain nothing but a data volume that can be mounted by the other containers that provide the applications. These data-only containers shall not be removed, but can persist over longer time, also if the application containers get updated.
 +
 +In the KnowRob use case, there is one container for user data (called user_data), and another one for common data (called knowrob_data). The user container will serve as sandbox in which users can store their files, the knowrob_data container will provide commonly used data sets. The common data container knowrob_data is [[https://docs.docker.com/userguide/dockerrepos/#automated-builds|auto-built]] on [[https://hub.docker.com/|DockerHub]] from the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GithHub repository.
  
-==== Creating the user data containers ==== 
-The user containers are built from the user_data Dockerfile. It basically just creates a user 'ros' in order to set the permissions correctly and creates the directory '/home/ros/sandbox'. The following commands build a Docker image from this Dockerfile (needs to be done only when the Dockerfile gets updated) and start a container, in this case for the user 'demo'. 
-  cd docker/user-sandbox 
-  docker build -t knowrob/user_data . 
-  docker run --name demo_data knowrob/user_data true 
  
 ==== Updating the common data container ==== ==== Updating the common data container ====
 The knowrob_data image is automatically built whenever new data is pushed to the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GitHub repository. This image is, however, not automatically updated in the server's registry. In order to pull the newest version and to replace the running container with a new one, use the following commands: The knowrob_data image is automatically built whenever new data is pushed to the [[https://github.com/knowrob/knowrob_data|knowrob_data]] GitHub repository. This image is, however, not automatically updated in the server's registry. In order to pull the newest version and to replace the running container with a new one, use the following commands:
  
 +<code bash>
   docker pull knowrob/knowrob_data:latest    docker pull knowrob/knowrob_data:latest 
   docker rm knowrob_data   docker rm knowrob_data
   docker run --name knowrob_data knowrob/knowrob_data:latest true   docker run --name knowrob_data knowrob/knowrob_data:latest true
 +</code>
 +
  
 ==== Starting a user container that includes the data volumes ==== ==== Starting a user container that includes the data volumes ====
-If the KnowRob image has been built as described earlier, it can be started using the following command such that both the common and the user-specific data volumes are mounted: 
  
 +If the KnowRob image has been built as described earlier, it can be started using the following command such that both the common and the user-specific data volumes are mounted:
 +<code bash>
   # interactive mode   # interactive mode
-  docker run -t -P -i \+  docker run -t -P -i --rm \
              --volumes-from knowrob_data \              --volumes-from knowrob_data \
-             --volumes-from demo_data \+             --volumes-from user_data \
              --name demo \              --name demo \
 +             --link mongo_db:mongo \
              knowrob/hydro-knowrob-interactive \              knowrob/hydro-knowrob-interactive \
              /bin/bash              /bin/bash
      
   # daemon mode   # daemon mode
-  docker run -t -P -d \+  docker run -t -P -d --rm\
              --volumes-from knowrob_data \              --volumes-from knowrob_data \
-             --volumes-from demo_data \+             --volumes-from user_data \
              --name demo \              --name demo \
 +             --link mongo_db:mongo \
              knowrob/hydro-knowrob-daemon              knowrob/hydro-knowrob-daemon
-   +  </code> 
-These examples are for the user 'demo', so 'demo' and 'demo_data' have to be replaced with the respective username. +These examples are for the user 'demo', so 'demo' has to be replaced with the respective username.
- +
-==== Starting MongoDB ==== +
- +
-  docker run -d -v /data/db --name mongo_data busybox true +
-  docker run -d -p 27017:27017 --volumes-from mongo_data --name mongo_db mongo +
- +
-==== Importing data into a MongoDB ==== +
- +
-If you want to import data into a dockerized MongoDB instance, first start an interactive container that mounts the host directory which contains the exported json files: +
-  docker run -t -P --rm --link mongo_db:mongo \ +
-         -v /home/tenorth/work/roslog:/var/roslog \ +
-         -i knowrob/hydro-knowrob-interactive:1.0.2 /bin/bash +
- +
-Then you can run the 'mongoimport' program from within the container. Make sure to set the host correctly, i.e. to refer to the MongoDB instance in the other container: +
-  mongoimport --host "$MONGO_PORT_27017_TCP_ADDR:$MONGO_PORT_27017_TCP_PORT" --db roslog --collection tf tf.json +
-   +
-==== Starting the web app in a container ==== +
- +
-The web app needs access to the Docker socket, so you have to start it like this: +
-  docker run --rm -i -p 5000:5000 -v /var/run/docker.sock:/var/run/docker.sock knowrob/webrob python webrob.py +
- +
-You can then access the Web interface at http://localhost:5000 +
- +
- +
-==== Pulling the containers manually ==== +
- +
-  docker pull knowrob/hydro-knowrob-daemon +
-  docker pull knowrob/webrob +
-  docker pull knowrob/knowrob_data +
-  docker pull knowrob/user_data+