Do not manually edit the grubenv file

[schaefi]

Make sure to use grub2-editenv for changing content in the grub env blob. This Fixes #2986

[Conan-Kudo]

Untested, but this might actually make it worse. Remember that the grub2-editenv behavior changes depending on what it detects the disk to be, and that means it will probably produce bad data unless we're running this at the very end when we're in the "real disk" mounted as loops and whatnot.

[schaefi]

Remember that the grub2-editenv behavior changes depending on what it detects the disk to be

Hmm, that I don't understand. You are saying grub2-editenv has logic to detect anything in the system ? I don't see this. It looks also code wise just like a tool that can read and modify contents of the grubenv binary blob.

What always bites me is the magic in e.g grub2-probe or other grub toolings (mkconfig) which takes information from the host it is called on which is always broken because an image build process doesn't run on the target host.

With regards to this change I don't see how grub2-editenv would make it worse

Thoughts ?

[Conan-Kudo]

grub2-editenv does exactly the same thing as the rest of the tools do. I am unsure of the effects run before we are in the create phase.

[schaefi]

I did a number of tests and I don't see any logic applied by grub2-editenv other than reading and manipulating a file. Here is some data

[root@fedora ~]# strace -e openat grub2-editenv create

openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/liblzma.so.5", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libdevmapper.so.1.02", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libudev.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/boot/grub2/grubenv.new", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
+++ exited with 0 +++

[root@fedora ~]# strace -e openat grub2-editenv - set root=bogus
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/liblzma.so.5", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libdevmapper.so.1.02", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libudev.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/boot/grub2/grubenv", O_RDONLY) = 3
openat(AT_FDCWD, "/boot/grub2/grubenv", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
+++ exited with 0 +++

no complains, you can give it anything unlike other tools like

[root@fedora ~]# strace -e openat grub2-probe /
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libdevmapper.so.1.02", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libudev.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/boot/grub2/device.map", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_MESSAGES/grub.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_MESSAGES/grub.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_MESSAGES/grub.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_MESSAGES/grub.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_MESSAGES/grub.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/grub.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/proc/self/mountinfo", O_RDONLY) = 3
grub2-probe: error: failed to get canonical path of `LiveOS_rootfs'.
+++ exited with 1 +++

then just list the contents

[root@fedora ~]# strace -e openat grub2-editenv list
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/liblzma.so.5", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libdevmapper.so.1.02", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libudev.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib64/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/boot/grub2/grubenv", O_RDONLY) = 3
root=bogus
+++ exited with 0 +++

and the hexdump

[root@fedora ~]# hexdump -C /boot/grub2/grubenv 
00000000  23 20 47 52 55 42 20 45  6e 76 69 72 6f 6e 6d 65  |# GRUB Environme|
00000010  6e 74 20 42 6c 6f 63 6b  0a 23 20 57 41 52 4e 49  |nt Block.# WARNI|
00000020  4e 47 3a 20 44 6f 20 6e  6f 74 20 65 64 69 74 20  |NG: Do not edit |
00000030  74 68 69 73 20 66 69 6c  65 20 62 79 20 74 6f 6f  |this file by too|
00000040  6c 73 20 6f 74 68 65 72  20 74 68 61 6e 20 67 72  |ls other than gr|
00000050  75 62 2d 65 64 69 74 65  6e 76 21 21 21 0a 72 6f  |ub-editenv!!!.ro|
00000060  6f 74 3d 62 6f 67 75 73  0a 23 23 23 23 23 23 23  |ot=bogus.#######|
00000070  23 23 23 23 23 23 23 23  23 23 23 23 23 23 23 23  |################|
*
00000400

So for me the editenv tool is not more than a data dumper

it supports:

• create blank new blob
• increment an integer value of some key
• list contents
• set key=value
• unset key=value

And all of this seems to manage just one file and no more logic was involved, at least in the tests I did.
I also wondered why it should do more

[Conan-Kudo]

It changed in GRUB 2.14: https://gitlab.freedesktop.org/gnu-grub/grub/-/commit/d6525f0e65332555e0c65d4846050a45ea51969e

[Conan-Kudo]

This patch exists in Fedora and SUSE grub2-editenv, but it is also in upstream GRUB 2.14.

[schaefi]

🤦 man why are people doing this. There is not even an option to skip it when it doesn't make any sense for example in an image build. The only way to circumvent that is by using a non standard path and copy the result back. This will also be a problem when we apply the optional environment variables

[Conan-Kudo]

To be fair, things like the env vars need to be in the external environment block if btrfs is used for /boot, so we shouldn't be writing those to a file anyway. GRUB reads both the file and the environment block AFAICT.

[schaefi]

I still think we should use grub2-editenv for any grubenv modifications, manually messing with the file is something we really should prevent.

I'm also pretty much convinced that the error reported in #2986 is not related to the proposed change here. The manual editing would only apply to the vendor grubenv file which would be searched in EFI/fedora and we don't touch /boot/grub2/grubenv at all unless a bootloader/environment setup exists in the image description, and in this case we would call grub2-editenv.

As such I don't see how kiwi can cause the broken grubenv

nevertheless I still believe the proposed change in this PR makes sense

Thoughts ?

[Conan-Kudo]

I don't think you're wrong, but I'd like to see it run through our staging first. This change makes me nervous and I'd like to be certain we don't screw up. 😅

[schaefi]

I fully agree, I zip a Staging build then we will see if anything fails.

btw, all latest rolling releases e.g. rawhide are still on grub 2.12. I guess we will experience failed builds once they upgrade to 2.14

[schaefi]

To be fair, things like the env vars need to be in the external environment block if btrfs is used for /boot, so we shouldn't be writing those to a file anyway. GRUB reads both the file and the environment block AFAICT.

I can't follow you with this. The external environment block is a 1k size file read by grub. grub reads this file via its load_env command. The actual reading via load_env can be done in several ways, as part of an embedded grub earlyboot script or as part of the grub config file after the earlyboot script. In either way you can pass --file to load_env to specify the location for reading it. Therefore it depends on the layout of the image what the correct load_env call is. Almost all distributions provides a pre-built grub image and packages it. Those usually also contains earlyboot setup to deal with the env blob for the desired layout, and then there is also the grub tooling magic in grub2-mkconfig which probes the layout and places the hopefully correct load_env.

From a kiwi perspective the only part we do when we talk about the env blob is calling grub2-editenv with all default settings such that the expectation is that it does the right thing according to the distribution it was provided for. Same concept as with any other grub tooling. We assume it does the right thing, find out that it's not true and add yet another fix_XXX method which I would like to get rid of rather today than tomorrow

Where am I wrong with this ?

[schaefi]

Staging build runs

[schaefi]

@Conan-Kudo All staging tests passed. I did a closer look on the integration test which does env blob adaptions:

• https://build.opensuse.org/package/show/Virtualization:Appliances:Images:Testing_x86:rawhide/test-image-live-disk
  The Virtual_btrfs profile

For me things are working as before

[schaefi]

But here is the funny thing, with the above image build I can see that env_block parameter

[root@fedora ~]# grub2-editenv list
saved_entry=54272cc365674dc1843e9a46af1404f9-7.0.0-62.fc45.x86_64
env_block=512+1
menu_auto_hide=1
boot_success=0
boot_indeterminate=0

kiwi only added the following

EXEC: [chroot /var/tmp/kiwi_mount_manager.yhn5db9i grub2-editenv - set menu_auto_hide=1 boot_success=1]

The other settings were either provided by some distro package or are part of the grub tooling

EXEC: [chroot /var/tmp/kiwi_mount_manager.yhn5db9i grub2-mkconfig -o /boot/grub2/grub.cfg]
EXEC: [chroot /var/tmp/kiwi_mount_manager.oww407bs grub2-install --skip-fs-probe --directory /usr/lib/grub/i386-pc --boot-directory /boot --target i386-pc --modules ext2 iso9660 linux echo configfile search_label search_fs_file search search_fs_uuid ls normal gzio png fat gettext font minicmd gfxterm gfxmenu all_video xfs btrfs squash4 lvm luks gcry_rijndael gcry_sha256 gcry_sha512 crypto cryptodisk test true loadenv multiboot part_gpt part_msdos biosdisk vga vbe chain boot /dev/loop0]

I cannot imagine another code part, and these settings env_block= and saved_entry= smells like grub BLS