Understand how linux containers works with practical examples

Nowadays a bast majority of server workloads run using linux containers because of his flexibility and lightweight but have you ever think how does linux containers works. In this tutorial we will demystify how does linux containers works with some practical examples. Linux containers works thanks two kernel features: namespaces and cgroups.

Linux Namespaces

A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource. Changes to the global resource are visible to other processes that are members of the namespace, but are invisible to other processes. One use of namespaces is to implement containers. [1]

Currently the linux kernel have 8 types of namespaces:

Linux control groups (cgroups)

Cgroups allow you to allocate resources — such as CPU time, system memory, network bandwidth, or combinations of these resources — among user-defined groups of tasks (processes) running on a system. You can monitor the cgroups you configure, deny cgroups access to certain resources, and even reconfigure your cgroups dynamically on a running system. [2]

Container Fundamentals (key technologies)

In this section we gonna make some practices with the following key technologies that make possible the usage of containers in linux:

NOTE: This tutorial was made using a VM with 1GB of ram and 1vCPU using debian 10 buster with kernel 4.19.0-16-amd64

Process namespace fundamentals

A process namespace isolate a running command from the host. Let’s see how to implement a process namespace in linux.

List process namespaces

Get the PID of the current terminal

Launch a new zsh terminal using namespaces

See the process tree from the parent

List namespaces

Filesystem — Overlay FS fundamentals

Containers need to have a filesystem, one of the most used filesystem for containers is overlay who can mount with layes and merge in a single directory, the lower layers are read only and all changes are made on the upper layer. Let's see how does overlay fs works.

Create directories

Create some files in lower directories

Create overlay FS

Create, modify files

Umount overlay fs

Inspect lower and upper dirs

Networking — Linux bridge fundamentals

Linux container uses network namespaces to isolate the network from the host, this is possible implementing a bridge interface that acts like network switch, and every container connect to that interface with his own ip address. Let’s see how does linux bridge and network namespaces works.

Create a Network Virtual bridge

List Network Interfaces

Assign an IP Address to bridge interface

Bring UP the bridge interface

Create 2 Network Namespaces

Create a Virtual Ethernet cable pair

Assign veth to namespaces

Assign IP address to veth within namespaces

Bring UP veth interfaces within Namespaces

Bring UP bridge veth in the local host

Configure default route within namespaces

Enable IP forward in the host

Configure MASQUERADE in the host for subnet

Control groups (cgroups) fundamentals

Control groups or cgroups are used by containers to limit the usage of resource in the host machine. Let’s see how does cgroups works.

Create cgroups directory

Mount cgroups directory

Create new directories under CPU controller

Assign CPU shares to every user (This example uses 1vCPU)

Create artificial load

Assign process to every user

Monitoring process

Create a container from scratch

So far we know how does linux namespaces works, now lets create a container using overlayfs, network namespaces, cgroups and process namespaces from scratch. Let’s see how a linux container is created.

Download and extract debian container fs from docker

Create bridge interface

Create overlay Filesystem from debian container fs

Mounting Virtual File Systems

Launch process namespace within fs/merged fs

Connect the container with br-cnt

Mount cgroup

Run commands within container

Clean all

Inspect Namespaces within a docker container

Fortunately for us there is a program that simplifies the usage of containers, for us this program is docker who manage the life-cycle of running a container. Let's see how does docker implement the namespaces running a container.

Install docker CE

Install docker community edition from official script in get.docker.com

Inspect Docker Network

Create a bridge network using docker

Inspect bridge network, see subnet using IP

Inspect Docker bridge network, see subnet using docker

Run an nginx web server

Inspect network namespace from nginx container

Create symlink from /proc to /var/run/netns

Check network interface within namespace

Check IP address of nginx container

Check port forwarding from 8080 to 80

Inspect cgroups in a docker container

Run a Ubuntu container with limited resources

See cgroup fs hierarchy

See attached task to container cgroup

Monitoring the container

Generate CPU load

Generate Memory load

Inspect overlay fs in a docker container

Run a ubuntu container with limited resources

NOTE: The merged layer is the actual container Filesystem

Inspect lower layers with tree and less

Inspect upper layer (It’s empty)

Run command withing the container

Inspect (again) upper layer (now it’s not empty)

Inspect docker process namespace

Run docker container

Launch process within container

See container tree process from container

List PID namespaces

See process using docker


In this tutorial we create our first container from scratch understanding what happen behind the scenes when we run a container. I hope this tutorial helps you to understand the technologies behind the linux containers.

Source code

Engineer || MSc student || DevOps in progress