Docker Guide PDF

Containers Containers u The developer worries about what’s inside the container: Code, libraries, package manager, apps, data u The Ops worry about what’s outside the container: Logging, remote access, monitoring, network config u All containers start, stop, copy, attach, migrate the same wa...

Containers Containers u The developer worries about what’s inside the container: Code, libraries, package manager, apps, data u The Ops worry about what’s outside the container: Logging, remote access, monitoring, network config u All containers start, stop, copy, attach, migrate the same way Docker u Build, ship & run any software any where u Docker is a tool designed to create, deploy, and run applications with ease by using containers u It allows developers to package an application with all the requirements such as libraries and other dependencies, ship it all as one package u It ensures that your application works seamlessly in any environment: Development, Test, or Production u Dockerized apps and dependencies can be shipped anywhere Docker u Docker file build a docker image which contains all the project code u You can run that image to create as many docker containers as you want u The created images can be uploaded on docker hub from where the image can be pulled and built in a container u Docker file à docker image à docker container à any environment What is an image? u An image is a text file with a set of pre-written commands, usually called as a docker file u Docker images are made up of multiple layers which are read-only filesystem u A layer is created for each instruction in a docker file and placed on top of the previous layer u When an image is turned into a container the docker engine takes the image and adds the read-write filesystem on top (as well as initializing various settings such as the IP address, name, ID, and resource limits) What is an image? Few basic commands u docker help u docker version u docker system info, docker system df, docker system df -v u docker images à displays a list of existing images in docker system u Repository u Tag u Image ID u Created u Size Few basic commands u u docker ps à displays the list of active containers u Container ID u Image u Command u Created u Status u Ports u Names docker ps -a à Display the list of all the container processes which are running or have run in the past Hello-World image u Search for an image that starts with “hello-world” from Docker Repository u u Pull the selected image from Docker Hub u u docker search hello docker pull hello-world Execute the “hello-world” pulled from Docker Repository u docker run hello-world Comparison between Docker and physical containers Containers advantages over VMs u Containers are lighter and smaller u Better resource utilization compared to VMs u Short boot-up process u Containerization is just virtualization at the OS level Virtualization u u Advantages: u Multiple OS in the same machine u Easy maintenance and recovery u Lower total cost of ownership compared to real machines Disadvantages: u Multiple VMs lead to unstable performance u Hypervisors are not as efficient as host OS u Long boot-up process Connection modes u u u Detached mode: u docker run –d ubuntu u d à detached mode Root User mode: u docker run –it ubuntu u i à interactive u t -à connected to terminal docker attach Examples u docker pull ubuntu à This pulls the ubuntu image from docker hub repository with the tag: latest u docker run –i ubuntu à This command helps you to get inside the container u docker ps u docker run –t ubuntu à This command calls a terminal from inside the container, this prevents the container from exiting u docker run –it ubuntu à This allows the container to run in the interactive mode as well as prevents it from exiting (crtl + P+ Q to get out of container without exiting) u docker run –d ubuntu à This allows the container to run a service in the background Lifecycle of a container Examples u docker run hello-world:latest u docker ps u docker ps – a u docker images u docker run –d –name=“container1” hello-world:latest u docker run –itd ubuntu u docker attach u docker stop u docker start u docker inspect u docker version u docker container run centos ping -c 5 127.0.0.1 u docker exec –it /bin/sh u docker logs Anatomy of Containers u Containers leverage a lot of features and primitives available in the Linux OS. The most important ones are namespaces and cgroups. All processes running in containers share the same Linux kernel of the underlying host operating system. This is fundamentally different compared with VMs, as each VM contains its own full-blown operating system. Architecture of Docker Namespaces The PID namespace is what keeps processes in one container from seeing or interacting with processes in another container. A process might have the apparent PID 1 inside a container, but if we examine it from the host system, it would have an ordinary PID, say 334: Control groups (cgroups) u Linux cgroups are used to limit, manage, and isolate resource usage of collections of processes running on a system. Resources are CPU time, system memory, network bandwidth, or combinations of these resources, and so on. Union filesystem (UnionFS) u The UnionFS forms the backbone of what is known as container images. u UnionFS is mainly used on Linux and allows files and directories of distinct filesystems to be overlaid and with it form a single coherent file system. u In this context, the individual filesystems are called branches. u Contents of directories that have the same path within the merged branches will be seen together in a single merged directory, within the new, virtual filesystem. When merging branches, the priority between the branches is specified. In that way, when two branches contain the same file, the one with the higher priority is seen in the final FS Runc u Runc is a lightweight, portable container runtime. It provides full support for Linux namespaces as well as native support for all security features available on Linux, such as SELinux, AppArmor, seccomp, and cgroups. u Runc is a tool for spawning and running containers according to the Open Container Initiative (OCI) specification. It is a formally specified configuration format, governed by the Open Container Project (OCP) under the auspices of the Linux Foundation. Containerd u Runc is a low-level implementation of a container runtime; containerd builds on top of it, and adds higher-level features, such as image transfer and storage, container execution, and supervision, as well as network and storage attachments. With this, it manages the complete life cycle of containers. Containerd is the reference implementation of the OCI specifications and is by far the most popular and widely-used container runtime. Sharing docker host data with containers u User -v option of docker run to mount a host volume into a container u Sharing a working directory from the host in a a certain directory in a container u Mount a working directory from the host into a certain directory in a container u Creating files or directories within the container, let the changes be written directly to the host working directory u docker run –it –v ~/dironhost:/dirinsidecontainer ubuntu Copying data to and from containers u Use docker cp command to copy a file form a working container to the docker host: u docker cp :/file1.txt u docker cp file1.txt container:/file1.txt Expose Ports u Use –p to map ports between the container and the container host u docker run -p 80:8080 tomcat Docker on Windows and Mac u Docker toolbox has been available for developers for few years u It precedes the newer tools such as Docker for Mac and Docker for Windows u The toolbox allows a user to work very elegantly with containers on any Mac or Windows computer. Containers must run on a Linux host. Neither Windows or Mac can run containers natively. Thus, we need to run a Linux VM on our laptop, where we can then run our containers. Docker Toolbox installs VirtualBox on our laptop, which is used to run the Linux VMs we need u you can only install Docker for Windows on Windows 10 Professional or Windows Server 2016 since it requires Hyper-V, which is not available on older Windows versions or on the Home edition of Windows 10. If you are using Windows 10 Home or an older version of Windows, you will need to stick with Docker Toolbox Base image u Docker base image is the basic image on which you add layers and create a final image containing your app u It keeps track of the difference between the base image and the new image by creating a new image layer using the union file system u Images are comprised of multiple layers u Every image contains a base layer u Docker uses a copy on write system u Layers are just read only images Layered filesystem A container image is made of a stack of immutable or read-only layers. When the Docker engine creates a container from such an image, it adds a writable container layer on top of this stack of immutable layers. Creating an image u The first one is by interactively building a container that contains all the additions and changes one desires and then committing those changes into a new image. u docker commit myubuntu:v1.0 u docker image history myubuntu:v1.0 u The second and most important way is to use a Dockerfile to describe what's in the new image and then build this image using that Dockerfile as a manifest. u Finally, the third way of creating an image is by importing it into the system from a tarball. u docker image save -o./myubuntu.tar myubuntu:v1.0 u docker image load –i./ myubuntu.tar Docker File u Docker file is the basic building block of docker containers u Docker file is a file with a set of instructions and forms the basis of any docker image u Everytime, base image is going to be based upon another image. You are going to pick up a base image and build on that image Various commands related to docker file u Env: is used to set one or more environment variables u Workdir: It defines the location where the command defined by CMD is to be executed u Run: It’s used to take the commands as it’s arguments and runs it to form an image u Volume: It’s used to enable access from your container to a directory u Add: It copies the file into the containers own file system from the source on the host at the stated destination u Expose: it’s used to expose the port to allow networking between the running process inside the container FROM u Every docker file starts with this command u It shows where the base image coming from u Will pick up an image from docker hub or some other repository and make some changes u Example: FROM ubuntu:latest MAINTAINER u It shows the maintainer or the owner of the docker file u It requires certain format – It requires the name and the email id u Format: MAINTAINER name u Example: FROM ubuntu:latest MAINTAINER yourname youremail Example From ubuntu:lastest MAINTAINER yourname youremail RUN apt-get update RUN apt-get -y install vim ENV u Environment variables in docker are declared with ‘ENV’ statement u Environment variables are notated in dockerfile as $variable_name or ${variable_name} u Set up the environment variable and pass a variable that we need to pass inside the container that runs on base image, example: ENV MYVALUE xyz u When you run the container, this value has to be passed using echo $MYVALUE Docker File – Continued u EXPOSE: good for inter-container communications - Ports are set up in the docker file to be exposed u EXPOSE 80 u CMD: command for starting up of a service of some kind u Example: From ubuntu:lastest MAINTAINER yourname youremail RUN apt-get update ENV MYVALUE xyx EXPOSE 80 CMD [“bin/bash”] Building a docker image u vi Dockerfile u docker build –t myubuntu:v1.0. u docker images Docker RUN vs Runtime (CMD & Entrypoint) Command u RUN executes commands in a new layer and creates a new image. It’s often used for installing software packages u CMD sets default command and/or parameters, which can be overwritten from command line when docker container runs u ENTRYPOINT configures a container that will run as an executable Example ENTRYPOINT ["/bin/echo", "Hello"] CMD ["world"] u docker run –it u u will produce “Hello world” docker run –it John u will produce “Hello John” Docker Hub u It’s a cloud based registry service which allows you to link to code repositories, build images and test them, stores manually pushed images and links to docker cloud, so you can deploy images to your hosts. u Publish/share an image on docker hub using the following steps: u Create an account on the docker hub u Login into the hub from docker host u Push your images Docker Hub u docker login à enter username and password u docker push yourrepo/image u docker tag hello-world yourrepo/hello-world u docker pull yourrepo/hello-world Docker registry u Deploy private registry: u docker run –d –p 5000:5000 –name registry registry:2 u docker image tag my-image localhost:5000/my-image u docker push localhost:5000/my-image Docker compose u It is a tool for defining & running multi-container docker applications u Use yaml files to configure application services (docker-compose.yml) u Can start all services with a single command (docker compose up) u Can stop all services with a single command (docker compose down) u Can scale up selected services when required u docker-compose –v Docker compose u docker-compose config u docker-compose up –d u docker-compose down u docker-compose –f docker-compose2.yml up Docker compose KITEMATIC u Instead of using the command line to manage your containers locally, you can use Kitematic UI which is a graphical interface. Docker Networking u Docker0 Bridge: u u Default bridge created by Docker to provide communication across Docker containers and external world including the host Host Network: u Host Network driver is used when isolation of container network stack from the docker host is not required u If a container is running on some port and the Host Network is being used, then application will be available on the same port on host’s IP address u It works only on Linux OS and not on Windows or Mac OS Docker Networking u Overlay: u Creates a distributed network and helps multiple docker daemons in communication u Allows secure communication u Helps docker swarm services to communicate u Below is the command to create overlay network in Docker u docker network create –d overlay mynetwork Docker Networking u u Macvlan: u Macvlan assign a Mac Address to a container which helps it to appear as a physical device on the network u This type of network is used by legacy applications, or applications which are supposed to be directly connected to physical network u Macvlan networks can be isolated by using various network interfaces None: u It helps to disable all the networking stack on a container u It is not available for swarm services u network –none flag is used while starting the container to completely disable the networking stack Docker Swarm u Docker Swarm is a cluster of machines, all running docker which provides scalable and reliable platform to run many containers. u With Swarm, IT admins and developers can establish and manage a cluster or Docker nodes as a single virtual system. Docker Swarm u Every Swarm has at least one manager (Generally, the one which is initialized first) u Port 2377 is the default port u Managers: u u Swarm0 Workers: u Swarm1 u Swarm2 u Swarm3. u docker swarm init --advertise-addr 10.0.0.1 u docker node ls TE5 u Write a docker file that builds an image to run your java classes in a container. You can pass the name of the class when starting the container. Build the image and push it on your docker hub account, then run it from your local machine. u Your Java classes should be hosted on your github account which you can reach from your docker image. u Try to use docker volumes, exposing the ports, docker compose in any simple useful scenario to show good understanding in using docker to run your Java applications u Capture that in 10- mins video and share the link when submitting your assignment