Should We Worry About Virtual Disk Fragmentation?

Recently, I explained about qcow2 or QEMU Copy-On-Write v2 and some performance trade-offs for virtual disks.

One thing that might alarm you is the appearance of a high percentage reported fragmentation on compressed qcow2 disks. Don’t worry, this isn’t what you probably think!

That’s all you really need to know, keep reading if you want to find out why.

A Lightly Used but Highly Fragmented Disk Image.

Here is another look at that test disk image when I first deployed it. A 16 GB disk has been allocated, only 3.1 GB has been used by the Debian Linux installation on it, and qcow2 compression has reduced the disk image file to just 1.1 GB:

host:# ls -lh
total 3.1G
-rw-r--r-- 1 root root 1.1G Feb  5 13:53 image.qcow2

host:# file image.qcow2
image.qcow2: QEMU QCOW Image (v2), 17179869184 bytes

host:# qemu-img info image.qcow2
image: image.qcow2
file format: qcow2
virtual size: 16G (17179869184 bytes)
disk size: 3.1G
cluster_size: 65536
Format specific information:
    compat: 0.10

host:# qemu-img check image.qcow2
No errors were found on the image.
47689/262144 = 18.19% allocated, 96.82% fragmented, 95.55% compressed clusters
Image end offset: 1168572416

I started the VM, connected to its console with virt-viewer, and ran some commands. I have trimmed the output to just keep the more interesting parts, and highlighted the most interesting of all. We’re currently using 2.7 GB, another 0.4 GB has been used and deleted, growing the allocated image to 3.1 GB:

vm:$ sudo bash
vm:# df -hT /
Filesystem                 Type  Size  Used Avail Used Mounted on
/dev/disk/by-uuid/2162...  ext4   16G  2.7G   12G  19% /

vm:# e2freefrag /dev/disk/by-uuid/2*
[...]

Min. free extent: 4 KB
Max. free extent: 1703936 KB
Avg. free extent: 29776 KB
Num. free extent: 438

[...]

vm:# e4defrag -c /
[...]

Total/best extents         68848/68476
Averages size per extent   36 KB
Fragmentation score        0
[0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]

Within the VM, there is effectively no fragmentation.

Now, I wanted to fragment the file system. I can’t simply copy /dev/zero into one monster file until I fill the file system, as that won’t lead to any significant fragmentation. I need to create a large number of files of various sizes scattered through a large directory tree.

I created and ran this script:

#!/bin/bash

rm -r landfill

WORKING=0
while [ "${WORKING}" == 0 ]
do
        TARGETDIR="landfill/${RANDOM}/${RANDOM}"
        mkdir -pv ${TARGETDIR}
        FILE=${TARGETDIR}/${RANDOM}
        SIZE=${RANDOM}
        echo "Creating ${SIZE} kbyte file ${FILE}"
        dd if=/dev/zero of=${FILE} bs=1k count=${SIZE}
        working=$?
done

Almost 6 minutes later my user account could no longer add files. I topped off the disk as root:

$ sudo bash
vm:# dd if=/dev/zero of=/tmp/zero bs=1M
vm:# ls -lh /tmp/zero
-rw-r--r-- 1 root root 784M Feb  5 15:16 /tmp/zero
vm:# df -hT /
Filesystem                 Type  Size  Used Avail Used Mounted on
/dev/disk/by-uuid/2162...  ext4   16G  16G      0 100% /

As it turns out, I still hadn’t caused any meaningful amount of fragmentation:

vm:# e4defrag -c /
[...]

Total/best extents         69925/69492
Averages size per extent   211 KB
Fragmentation score        0
[0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]

Let’s clean up the mess and shut down:

vm:# rm -r /tmp/zero landfill
poweroff

How does the disk image file look now?

host:# ls -lh
total 14G
-rw-r--r-- 1 root root 14G Feb  5 15:20 image.qcow2

host:# qemu-img info image.qcow2
image: image.qcow2
file format: qcow2
virtual size: 16G (17179869184 bytes)
disk size: 14G
cluster_size: 65536
Format specific information:
    compat: 0.10

host:# qemu-img check image.qcow2
No errors were found on the image.
251034/262144 = 95.76% allocated, 16.67% fragmented, 14.75% compressed clusters
Image end offset: 14892924928

Look what happened to the reported fragmentation! That isn’t the fragmentation within the VM’s file system, and it isn’t the fragmentation of the disk image itself.

Let’s use the convert operation to turn the almost completely allocated (but currently only lightly used) disk image into a new image file. We show you how to do various disk image conversions in Learning Tree’s Linux virtualization course.

host:# qemu-img convert -p -c -O qcow2 image.qcow2 image-new.qcow2

host:# ls -lh
total 15G
-rw-r--r-- 1 root root 1.1G Feb  5 15:30 image-new.qcow2
-rw-r--r-- 1 root root  14G Feb  5 15:20 image.qcow2

host:# qemu-img info image-new.qcow2
image: image-new.qcow2
file format: qcow2
virtual size: 16G (17179869184 bytes)
disk size: 1.0G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false

host:# qemu-img check image-new.qcow2
No errors were found on the image.
47508/262144 = 18.12% allocated, 97.77% fragmented, 97.02% compressed clusters
Image end offset: 1113915392

What’s the lesson?

Don’t worry about fragmentation reported by qemu-img!

If you are concerned about virtual machine performance, don’t use the qcow2 format, and certainly not that plus compression! Go with a raw file, or better yet, a dedicated device.