How Does Linux Boot? Part 4: Rescuing a System with the Grand Unified Boot Loader

Last week our detailed tour of the Linux boot sequence (which started here) got as far as the “shim” component handling Secure Boot in the UEFI environment and then calling the next program in the chain, probably grubx64.efi.

We found that grubx64.efi is part of the grub2-efi package and it references the file grub.cfg, also in the EFI System Partition:

# rpm -qf /boot/efi/EFI/redhat/grubx64.efi
# strings /boot/efi/EFI/redhat/grubx64.efi | grep grub.cfg
# tree /boot/efi
+-- EFI
    +-- BOOT
    |   +-- BOOTX64.EFI
    |   +-- fallback.efi
    +-- redhat
        +-- BOOT.CSV
        +-- fonts
        |   +-- unicode.pf2
        +-- gcdx64.efi
        +-- grub.cfg
        +-- grubx64.efi
        +-- MokManager.efi
        +-- shim.efi
        +-- shim-redhat.efi

4 directories, 10 files

So we have reached GRUB or the Grand Unified Boot Loader. Yes, its name is a playful reference to the Grand Unified Theory of physics.

What is Grub?

GRUB started as an ambitious project to create bootloader software that could load and start pretty much any operating system on any hardware.

Reality set in, the focus narrowed, and the feature set grew. It has become the de facto standard Linux and Solaris boot loader on Intel/AMD IA-32/x86-64 hardware. GRUB 2 is big and complicated, its manual is 130 pages long in PDF form.

The configuration file has grown from just ten lines or less in legacy GRUB to a behemoth of over 120 lines in GRUB version 2. The good news is that the GRUB package includes maintenance programs that are automatically invoked when you install an updated kernel package or when you build and install a new kernel from source, something we teach you how to do in Learning Tree’s Linux server administration course.

GRUB can read file systems, meaning that we finally reach the point in the story where the Linux kernel, stored in a Linux-native file system, makes its first appearance. GRUB finds the kernel, typically stored as /boot/vmlinuz-RELEASE, and loads it into RAM. It then starts the kernel, passing it a list of parameters.

The kernel needs to know how to find the root file system where it can find the directories containing crucial binaries (such as /bin and /sbin), shared libraries (/lib and /lib64), and configuration files (/etc). The problem is that we can’t predict the order in which devices will be detected, and so we can’t predict the names of the disk devices.

Worse yet, the disk with the root file system is probably attached through a SATA or SCSI controller. However, the kernel probably doesn’t have the needed device driver compiled into the monolithic core of the kernel — those device drivers are typically compiled as loadable modules and stored in a subdirectory of /lib/modules on the root file system.

We have a chicken-and-egg problem! The loadable module needed to read the disk is stored on that disk.

We solve that problem by having GRUB tell the kernel how to find something called an initial RAM disk image, a large file stored in the same area as the kernel itself. That special file contains a compressed archive of a file system that contains some loadable kernel modules, a few basic device-special files (such as /dev/null and /dev/{stdin,stdout,stderr}), and a few other basic pieces.

The kernel uncompresses the initial RAM disk image into RAM, mounts it as a file system, and uses the tools found there. Now it can find and mount the real file system in place of that initial image.

We still have the problem of not being able to predict the disk discovering and naming order. The solution is to instead pass something like these two parameters:

root=UUID=62dfc4a4-86c2-4ebf-aaa3-442ecc740122 ro

There is a root file system somewhere. We can’t tell the kernel where it is, but we can tell it what to look for. It will be in a partition containing a file system whose header includes the specified UUID. The “ro” tells the kernel to initially mount this file system in read-only mode so any needed initial file system consistency checks can be done safely.

Normally you turn on the system and let it boot up. But a corrupted system may not be able to start. I recently had to deal with a system where a series of misadventures had left it in a horrible state. Its kernel supported NSA Security-Enhanced Linux or NSA SELinux. It was configured for strict enforcement of a targeted policy. However, the misadventures had applied some inappropriate security labels to the files containing the security policy! Furthermore, an empty file named /.autorelabel had been created, forcing an SELinux relabeling of the entire file system.

So… If you try to start a strict SELinux operating system that wants to start by relabeling its entire file system using an SELinux policy that it doesn’t allow itself to read, it doesn’t get any further than printing one kernel message on the console announcing that it can’t read the needed policy and so it’s freezing.

GRUB to the rescue!

Reset the system and press e to edit the default bootable kernel definition before the timer counts down. Arrow down to the line specifying the Linux kernel file and its parameters, arrow to the end of the line, and add “S selinux=0“. The S means to go to the single-user or maintenance state instead of the default run level, and the second parameter disables SELinux. Now press Ctrl-X to boot with those new parameters. When the system comes up, remove that file calling for automatic relabeling and then reboot.

# rm /.autorelabel
# reboot

Get into the GRUB boot menu again, and this time add “S enforcing=0” SELinux functionality will be enabled, but it will not enforce the policy. This allows you to boot up to single-user mode and do SELinux repairs without the SELinux restrictions. Now you can manually relabel the file system:

# restorecon -vR /

See /usr/src/linux/Documentation/kernel-parameters.txt in the kernel source documentation on your system or online for many more things you can pass through GRUB to save the day.

Next week, things get really interesting with something new and big called system!

Type to search

Do you mean "" ?

Sorry, no results were found for your query.

Please check your spelling and try your search again.