Study Topic: Docker

From Matt Morris Wiki
Jump to navigation Jump to search

This is a Study Leave topic.

What Is It?

Current hottest containerisation tech

Why Study It?

Looking to use this to distribute work onto AWS etc

Toggl code

Toggl code is TECH-DOCKER

Deliverables

Have worked through all examples in book - a combination of package build times and a couple of road blocks meant it was something like 15 housr elapsed time in all.

Only bits not done were all swarm/consul examples and most of the Remote API exmaples (I just verified I could connect via netcat locally and get json back).

WriteUp

  • DONE install boot2docker 1.6.2 on Mac
  • DONE get a RHEL instance running, or maybe Debian, to get docker native as well
  • DONE I have Docker Hub login as mattmorris (home email, new password)
  • DONE Work through "The Docker Book"

Docker seems to be trying to get the Hub in the middle of everyone's build processes. Is that really a good thing? Have a look here for anti-Docker voices

CentOS Installation

See Docker Installation: CentOS

Base Install

Make sure your existing packages are up-to-date:

  • $ sudo yum update

Download the Docker RPM to the current directory.

Use yum to install the package.

  • $ sudo yum localinstall --nogpgcheck docker-engine-1.7.0-1.el7.centos.x86_64.rpm

Start the Docker daemon.

  • $ sudo service docker start

Verify docker is installed correctly by running a test image in a container.

  • $ sudo docker run hello-world

Creating a "docker" group

Creating a "docker" group avoids having to use sudo with docker by giving read/write access to the relevant Unix socket.

There are security implications to this, see here for more details: Docker Daemon Attack Surface.

To create the docker group and add your user:

  • Log into Centos as a user with sudo privileges.
  • Create the docker group and add your user.
    • $ sudo usermod -aG docker your_username
  • Log out and log back in.
    • This ensures your user is running with the correct permissions.
  • Verify your work by running docker without sudo.
    • $ docker run hello-world

Start the docker daemon at boot

To ensure Docker starts when you boot your system, do the following: 

 $ sudo chkconfig docker on

Further customisation

If you need to add an HTTP Proxy, set a different directory or partition for the Docker runtime files, or make other customizations, read the docker Systemd article to learn how to customize your Systemd Docker daemon options.

Uninstall

You can uninstall the Docker software with yum.

List the package you have installed:

  • $ yum list installed | grep docker
    docker-engine.x86_64                1.7.0-1.el6
                                                                                                                                                             @/docker-engine-1.7.0-1.el6.x86_64.rpm

Remove the package:

  • $ sudo yum -y remove docker-engine.x86_64

This command does not remove images, containers, volumes, or user-created configuration files on your host.

To delete all images, containers, and volumes, run the following command:

  • $ rm -rf /var/lib/docker

Additionally, locate and delete any user-created configuration files.

Basics

in boot2docker

  • boot2docker ip to get VM IP

Can get graphical management via DockerUI

  • docker run -d -p 9000:9000 --privileged -v /var/run/docker.sock:/var/run/docker.sock dockerui/dockerui
  • then http://<boot2docker host ip>:9000

In docker

  • docker ps to see containers
  • docker --name NAME to give name
  • docker run, give -d for daemon mode
    • docker run --name bob_the_daemon -d ubuntu /bin/bash -c "while true; do echo hello world; sleep 1; done"
    • can use --restart=always or --restart=on-failure:n to restart automatically if container fails
  • docker start/stop/restart for process control
  • docker kill if want SIGKILL rather than SIGTERM
  • docker rm to remove
    • can do docker rm 'docker ps -a -q' to remove all
  • docker attach for consoles
  • docker top to see what's running in a container
  • docker stats for quick CPU/mem/IO metrics
  • docker exec to run something inside a container (can be interactive)
    • docker exec -it container_name /bin/bash
  • docker inspect to get Go template dump of information about the container's setup
  • docker logs, can have -f for monitoring, -t for timestamps, --tail 0 to just get new
    • run --log-driver to select other logging modes from default (json-file), might pick "syslog", or "none" to deactivate
  • use docker run -p to specify ports, e.g. docker run -d -p 8080:80 --name static_web mattmorris/static_web nginx -g "daemon off;"
  • docker port container_name port_number to see what a port is mapped to, e.g. docker port static_web 80

Dockerfile Commands

ADD source destination

  • Adds file into image at destination location
  • Source can be URL, e.g. ADD http://wordpress.org/latest.zip /root/wordpress.zip
  • Will automatically unpack tars, e.g. ADD latest.tar.gz /var/www/wordpress/
  • Will create target directories if not already present

COPY source destination

  • Copies files - does not have extraction/decompression capabilities
  • Can copy directories, e.g. COPY conf.d/ /etc/apache2/
  • Source must be relative to build context, i.e. Dockerfile directory
  • Will create target directories if not already present

CMD [program, p1, p2, ...]

  • Runs a command - will be overriden by anything specified in "docker run"
  • CMD ["/bin/true"]
  • CMD ["/bin/bash", "-l"]

ENTRYPOINT [program, p1, p2, ...]

  • Provides a command, taking any "docker run" parameters as arguments
  • ENTRYPOINT ["/usr/sbin/nginx"]
  • ENTRYPOINT ["/usr/sbin/nginx", "-g", "daemon off;"]
  • Can combine with CMD in order to get defaultable parameters
  • Can override at runtime by --entrypoint flag

ENV name value

  • Sets environment variables
  • ENV NAME1=value1 NAME2=value2 ...
  • Can reuse in other commands, e.g. WORKDIR $NAME1
  • Will be persisted into containers
  • Can also be passed at runtime by -e flag

ONBUILD trigger commands

  • These are executed when the image is used as base of another image
  • It's as if the commands are added right after the FROM instruction
  • Can be any build instruction, e.g.
    • ONBUILD ADD . /app/src
    • ONBUILD RUN cd /app/src && make
  • Can see from docker inspect in "OnBuild" section

USER username

  • What user to run the image as
  • Can override at runtime by -u flag

VOLUME [vol1, vol2, ...]

  • Allows common mount points between containers
  • See Volumes

WORKDIR path

  • Sets current working directory
  • Can override at runtime by -w flag

Images

Building good docker images (Jonathan Bergknoff, 03Oct14)

  • Base images off of debian in preference to ubuntu
    • ubuntu:14.04 is 195 MB while debian:wheezy is 85 MB, but Ubuntu value not clear.
  • Don't install build tools without good reason
    • Some good reasons: you need specific versions or compile options
  • Don't leave temporary files lying around
    • it seems that if you leave a file on disk between steps in your Dockerfile, the space will not be reclaimed when you delete the file. It is also often possible to avoid a temporary file entirely, just piping output between commands.
  • Clean up after the package manager
    • If you run apt-get update in setting up your container, it populates /var/lib/apt/lists/ with data that's not needed once the image is finalized. You can safely clear out that directory to save a few megabytes.
  • Pin package versions
    • Minimise impact of external sate: pin package versions when updating through a package manager. E.g. # apt-get install redis-server=2:2.4.14-1
  • Combine commands
    • Fewer build cache steps. Backslashes \ help you out here for readability
  • Use environment variables to avoid repeating yourself
    • You can define environment variables with ENV and then reference them in subsequent RUN commands. Below, I've paraphrased an excerpt from the linked Dockerfile:

Using images:

  • docker pull to grab something
    • default tag is :latestd
  • docker images to list what we have locally
    • docker images repository to just get stuff for that repo, e.g. docker images ubuntu
  • docker search to look for images with that repo name, e.g. docker search puppet
  • docker commit can be used to build new images, but better to use "build" command
  • docker build can be used with a build file
    • docker build -t="repository:tag" path_to_docker_build_file
    • use docker run -it image name /bin/bash to diagnose intermediate images (----> lines)
    • --no-cache to force refresh
    • use ENV VARNAME VARVALUE & change VARVALUE to force refresh in subsequent lines
      • Debian may want to follow with "apt-get -qq update" (the qq is just a quite mode indicator), Fedora "yum -q makecache"

Docker Book Notes

Docker in Docker Notes

The Jenkins Docker-in-Docker examples in Ch 5 of the Docker book fail with

  • time="2015-09-18T10:23:18.393746164Z" level=error msg="There are no more loopback devices available."
  • time="2015-09-18T10:23:18.393846152Z" level=fatal msg="Error starting daemon: error initializing graphdriver: loopback mounting failed"

Apparently loopback exhaustion is a problem with docker-in-docker: build is fragile to loopback exhaustion

You can run this via "sudo bash scriptname" on the host machine before running docker 

#!/bin/bash

for i in {0..12}
do
    mknod -m0660 /dev/loop$i b 7 $i
done

Worked on CentOS 7.1, where there were 6 loopback devices before the script was run.

Jenkins and Docker

Ch 5 of the Docker Book shows examples for setting up Jenkins & Docker-in-Docker for single and multiple operating systems.

  • Docker-in-Docker apparently has issues - in CentOS I needed to add loopback devices to get things to work
  • The debian build of the multi-systems example fails! But both centos and ubuntu work so I'm not inclined to investigate.

Getting local container logs

This shows how to get logs out of a container volume. In this case, we use "--volumes-from" to access volumes in container "redis_primary". We attach a simple "ubuntu" image and run "cat" in it to extract. The "--rm" flag removes our docker container once it's done.

  • docker run -ti --rm --volumes-from redis_primary ubuntu cat /var/log/redis/redis-server.log

CentOS and linked components

Really nasty one. Docker 1.7 doesn't deal correctly with firewalld, the CentOS7 replacement for iptables. So trying the redis cluster example fails the secondary with a "SYNC: no route" error. The answer is to add an extra iptables entry once docker + firewalld are started but BEFORE containers are started:

  • iptables -A DOCKER -p tcp --dport 27017 -i docker0 -o docker0 -j ACCEPT

This is fixed in Docker 1.8 (not released as of 21 Sep 15)

See (eg): https://github.com/docker/docker/issues/16193

docker-compose Github location wrong

The curl to github for downloading Docker compose is docker/compose, not docker/docker-compose!

calling remote API from local machine

Example to fetch images - uses netcat to communicate with socket directly

  • echo -e "GET /images/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock

Objections To Docker

Containers Ain't All That

In Docker, Wouter has a philosophical issue with Docker's compartmentalisation - it can end up punting the problem of software compatibility by breaking the system up into too many atomic containers.

In The sad state of sysadmin in the age of containers, Erich Schubert worries about the possibility of containers stopping people thinking about what they are installing, and so running risks that they do not understand: "Docker is the new 'curl | sudo bash'".

In Docker vs Reality , 0 - 1, Kris Buytaert reckons that immutable applications are a smaller part of real-life problems than many people seem to think - so while Docker is good at the immutable part, the excitement about it is more hype than grounded in reality.

Containers Yes, Docker No

In LXC containers are awesome, but Docker.io sucks, Cal Leeming reckons that Docker botches LXC, but "For some of you, Docker might be the best step forward especially if you do not have the in-house skills to create your own application container stack."

In Let's review.. Docker (again), Cal Leeming raises objections to Dockerfile structure, the poor practical performance of layers, Docker Hub constraints, security setup, kernel incompatibility issues.

In The case against Docker, Andreas Jung found docker image builds slow and flaky, and cleanups more hassle than they should be - the overall effect being to make any overall advantage of Docker very unclear.

In Of Containers, Dockers, Rockets, and Daemons, Maciej Pasternacki describes Docker as implementation-defined, monolithic and opaque - and is hopeful that CoreOS/Rocket might do better.

Rocket Rival

Towards the end of 2014, CoreOS came up with their own container runtime, Rocket. Their design drivers are:

  • Composable. All tools for downloading, installing, and running containers should be well integrated, but independent and composable.
  • Security. Isolation should be pluggable, and the crypto primitives for strong trust, image auditing and application identity should exist from day one.
  • Image distribution. Discovery of container images should be simple and facilitate a federated namespace, and distributed retrieval. This opens the possibility of alternative protocols, such as BitTorrent, and deployments to private environments without the requirement of a registry.
  • Open. The format and runtime should be well-specified and developed by a community. We want independent implementations of tools to be able to run the same container consistently.
Retrieved from "http://www.matthew-morris.com/wiki/index.php?title=Study_Topic:_Docker&oldid=166303"