How To Use Linux Containers (LXC) With LVM

800px-Container_shipLast week I was explaining how Logical Volume Management or LVM was useful for file system maintenance on Linux. Containers are all the rage on Linux now, and LVM supports them very nicely. Let’s see how to do that!

First, though, what are Linux Containers or LXC?

LXC or a container is a virtualization environment. You can run programs within a container, which is an isolated Linux system running on a Linux host controlling several such containers. The concepts and capabilities are similar to Solaris Containers and AIX Workload Partitions. For lots more detail on containers and other Linux-based virtualization technologies, check out the new Linux Virtualization course that will start running in the next few months.

Containers provide resource isolation. You can limit the container’s use of CPU, memory, block I/O, and network bandwidth, and change these while the container is running. Within the container you see only its file system tree and a unique set of processes and user identities.

The controlling host sees each container’s file system as a hierarchy beneath /var/lib/lxc/containername/rootfs/, and it sees and can signal the processes running within the container.

The container starts very quickly, no more than a few seconds, because it does not need to load and start a kernel. It uses the kernel already running on the host. There is an init (these days, really systemd) process running as PID 1 within the container, and pstree in the container would show all of its processes running as descendants of that PID 1.

However, when you run pstree on the host, you see that second init running within the container. It and all of its descendants have different PIDs as seen from the host.

Each of these containers has its own file system, but you don’t see them in df output on the host as they aren’t mounted.

Exploring a Linux Container

Let’s say you have just 2 GB free on your root file system, where you have already installed a container named mytask. On the host you would see something like this:

# ls -lF /var/lib/lxc
total 0
drwxrwx---. 3 root root 44 Feb 29 10:21 mytask/
# ls -lF /var/lib/lxc/mytask
total 8
-rw-r--r--. 1 root root 2543 Feb 29 10:21 config
-rw-r--r--. 1 root root    0 Feb 29 10:21 fstab
drwxr-xr-x. 1 root root   22 Feb 29 10:21 rootfs/
# ls -F /var/lib/lxc/mytask/rootfs
bin/   dev/  home/  lib64/  mnt/  proc/  run/   selinux/  sys/  usr/
boot/  etc/  lib/   media/  opt/  root/  sbin/  srv/      tmp/  var/
# df -h
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/rhel-root   8.5G  6.5G  2.0G   3% /
devtmpfs                3.9G     0  3.9G   0% /dev
tmpfs                   3.9G  140K  3.9G   1% /dev/shm
tmpfs                   3.9G  8.9M  3.9G   1% /run
tmpfs                   3.9G     0  3.9G   0% /sys/fs/cgroup

The container’s root file system is in that subdirectory rootfs, which is within the hosts’s root file system.

But now ask within that container for the list of all file systems:

# df -h
Filesystem              Size  Used Avail Use% Mounted on
rootfs                  8.5G  6.5G  2.0G   3% /
/dev/mapper/rhel-root   8.5G  6.5G  2.0G   3% /
tmpfs                   798M   12k  798M   1% /run
tmpfs                   5.0M     0  5.0M   0% /run/lock
tmpfs                   1.8G     0  1.8G   0% /run/shm

You can further verify what’s going on by running this on the host:

# ls -id /var/lib/lxc/mytask/rootfs
8332512 /var/lib/lxc/mytask/rootfs

and this within the running container:

# ls -id /
8332512 /

Ahah! The same i-node, they’re the same directory. But within the container, that directory is the root of the file system:

# cd /
# ls -lid . ..
8332512 drwxr-xr-x. 22 root root 4096 Feb 27 10:37 .
8332512 drwxr-xr-x. 22 root root 4096 Feb 27 10:37 ..

There is no higher “..“, we can’t climb up out of the container’s file system.

You can simply copy files from the host into the container’s file system, and within the container you can do whatever you want (except escape, if the container is set up properly). From the host you can inspect the container’s file system for malware, and you can easily back up one or all containers.

Linux Containers and LVM

This looks very useful! Your enthusiasm for containers can quickly fill that root file system on the host.

One solution is to use the flexibility of LVM or Logical Volume Management to simply expand the host’s root file system. However, wouldn’t it make more sense to create a new file system on top of LVM and mount that as /var/lib/lxc? Then your file system support for container activity is separated from maintaining the host OS.

But let’s take it a step further — why not use LVM to maintain one logical volume per container and mount each of those as subdirectories of /var/lib/lxc? Each volume could be named for the container, or for the project supported by that container, or whatever you want. Storage use for that container is limited to the size of its logical volume.

The very nice thing is that LVM lets you do whatever makes the most sense to you for solving your problem!

To learn more, have a look at our complete curriculum of UNIX and Linux Training courses including 5 new 1-day online sessions!

Type to search

Do you mean "" ?

Sorry, no results were found for your query.

Please check your spelling and try your search again.