Last week I was explaining how Logical Volume Management or LVM was useful for file system maintenance on Linux. Containers are all the rage on Linux now, and LVM supports them very nicely. Let’s see how to do that!
LXC or a container is a virtualization environment. You can run programs within a container, which is an isolated Linux system running on a Linux host controlling several such containers. The concepts and capabilities are similar to Solaris Containers and AIX Workload Partitions. For lots more detail on containers and other Linux-based virtualization technologies, check out the new Linux Virtualization course that will start running in the next few months.
Containers provide resource isolation. You can limit the container’s use of CPU, memory, block I/O, and network bandwidth, and change these while the container is running. Within the container you see only its file system tree and a unique set of processes and user identities.
The controlling host sees each container’s file system as a hierarchy beneath /var/lib/lxc/containername/rootfs/, and it sees and can signal the processes running within the container.
The container starts very quickly, no more than a few seconds, because it does not need to load and start a kernel. It uses the kernel already running on the host. There is an init (these days, really systemd) process running as PID 1 within the container, and pstree in the container would show all of its processes running as descendants of that PID 1.
However, when you run pstree on the host, you see that second init running within the container. It and all of its descendants have different PIDs as seen from the host.
Each of these containers has its own file system, but you don’t see them in df output on the host as they aren’t mounted.
Let’s say you have just 2 GB free on your root file system, where you have already installed a container named mytask. On the host you would see something like this:
# ls -lF /var/lib/lxc total 0 drwxrwx---. 3 root root 44 Feb 29 10:21 mytask/ # ls -lF /var/lib/lxc/mytask total 8 -rw-r--r--. 1 root root 2543 Feb 29 10:21 config -rw-r--r--. 1 root root 0 Feb 29 10:21 fstab drwxr-xr-x. 1 root root 22 Feb 29 10:21 rootfs/ # ls -F /var/lib/lxc/mytask/rootfs bin/ dev/ home/ lib64/ mnt/ proc/ run/ selinux/ sys/ usr/ boot/ etc/ lib/ media/ opt/ root/ sbin/ srv/ tmp/ var/ # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/rhel-root 8.5G 6.5G 2.0G 3% / devtmpfs 3.9G 0 3.9G 0% /dev tmpfs 3.9G 140K 3.9G 1% /dev/shm tmpfs 3.9G 8.9M 3.9G 1% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
The container’s root file system is in that subdirectory rootfs, which is within the hosts’s root file system.
But now ask within that container for the list of all file systems:
# df -h Filesystem Size Used Avail Use% Mounted on rootfs 8.5G 6.5G 2.0G 3% / /dev/mapper/rhel-root 8.5G 6.5G 2.0G 3% / tmpfs 798M 12k 798M 1% /run tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 1.8G 0 1.8G 0% /run/shm
You can further verify what’s going on by running this on the host:
# ls -id /var/lib/lxc/mytask/rootfs 8332512 /var/lib/lxc/mytask/rootfs
and this within the running container:
# ls -id / 8332512 /
Ahah! The same i-node, they’re the same directory. But within the container, that directory is the root of the file system:
# cd / # ls -lid . .. 8332512 drwxr-xr-x. 22 root root 4096 Feb 27 10:37 . 8332512 drwxr-xr-x. 22 root root 4096 Feb 27 10:37 ..
There is no higher “..“, we can’t climb up out of the container’s file system.
You can simply copy files from the host into the container’s file system, and within the container you can do whatever you want (except escape, if the container is set up properly). From the host you can inspect the container’s file system for malware, and you can easily back up one or all containers.
This looks very useful! Your enthusiasm for containers can quickly fill that root file system on the host.
One solution is to use the flexibility of LVM or Logical Volume Management to simply expand the host’s root file system. However, wouldn’t it make more sense to create a new file system on top of LVM and mount that as /var/lib/lxc? Then your file system support for container activity is separated from maintaining the host OS.
But let’s take it a step further — why not use LVM to maintain one logical volume per container and mount each of those as subdirectories of /var/lib/lxc? Each volume could be named for the container, or for the project supported by that container, or whatever you want. Storage use for that container is limited to the size of its logical volume.
The very nice thing is that LVM lets you do whatever makes the most sense to you for solving your problem!
To learn more, have a look at our complete curriculum of UNIX and Linux Training courses including 5 new 1-day online sessions!
