Booting A/B, where is the kernel used for booting device?

[AndersGnistrup]

I have a working system so this is just a question about "how stuff works".
I am using branch scarthgap-l4t-r35.x on tegra-demo-distro and have found similar result for other branches. The development board is a orin-nano 8Gb NVidia development board. That branch was selected to get back to a known state.

I have inspecting the partition layout and from the file flash_l4t_t234_nvme_rootfs_ab.xml there should be a redundant setup. That also seems to match the logs during flashing

== Step 4: Writing partitions on external storage device at 2024-06-24T07:04:57+02:00 ==
Waiting for USB storage device nvme0n1 from a92bfe0a...[/dev/sda]
Creating partitions
  [03] name=A_kernel start=0 size=262144 sectors
  [04] name=A_kernel-dtb start=0 size=1536 sectors
  [05] name=A_reserved_on_user start=0 size=64768 sectors
  [06] name=B_kernel start=0 size=262144 sectors
  [07] name=B_kernel-dtb start=0 size=1536 sectors
  [08] name=B_reserved_on_user start=0 size=64768 sectors
  [09] name=recovery start=0 size=163840 sectors
  [10] name=recovery-dtb start=0 size=1024 sectors
  [11] name=esp start=0 size=131072 sectors
  [12] name=recovery_alt start=0 size=163840 sectors
  [13] name=recovery-dtb_alt start=0 size=1024 sectors
  [14] name=esp_alt start=0 size=131072 sectors
  [15] name=UDA start=0 size=819200 sectors
  [16] name=reserved start=0 size=982016 sectors
  [01] name=APP start=0 size=29360128 sectors
  [02] name=APP_b start=0 size=29360128 sectors

The APP has id=1 and APP_b has a id=2 so it seems to match the xml partition layout description in the xml file.

So far, so good, but I had a hard time to verify the boot selection/boot order using the boot log and system inspection.
So I decided to clear the A_kernel partition on a running system using dd if=/dev/zero of=/dev/nvme0n1p3.
I expected a boot failure/switch to B_kernel since the kernel was gone. But, the system booted without any delay.
Then I cleared the B_kernel, to see if some magic fallback had occurred. But, the system still booted without any delay.
Finally, I tried to clear the APP and that "worked". After three boot attempts in UEFI it switched to APP_b.

root@jetson-orin-nano-devkit-nvme:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
none                   3681192         0   3681192   0% /dev
/dev/nvme0n1p2        13655048    776516  12128148   6% /
tmpfs                  3815308         0   3815308   0% /dev/shm
tmpfs                  1526124      9668   1516456   1% /run
tmpfs                  3815308         0   3815308   0% /tmp
tmpfs                  3815308         8   3815300   0% /var/volatile
/dev/nvme0n1p11          64511       130     64382   0% /boot/efi
/dev/nvme0n1p11          64511       130     64382   0% /opt/nvidia/esp
tmpfs                   763060         4    763056   0% /run/user/0

But, the system should not boot at all since there is no kernel ?
Is something "not quite ready" but layout is kept for future use.
The running kernel is not in in sda3 but "somewhere else", probably in the rootfs partition?

[madisongh]

With the scarthgap-based, and later, branches, we default to using extlinux.conf for locating the kernel and device tree. The separate kernel/kernel-dtb partitions, if present in your flash layout, should still be kept up-to-date, since they are used as fallback by the L4TLauncher EFI application if it has problems loading the OS via the extlinux configuration.

[AndersGnistrup]

Thanks, I had the idea that the tegra-demo-distro did set all up, including the boot of A/B kernel.
It does not, the actual implementation is left to the "designer". It does makes sense to keep the tegra-demo-distro simple.

What it does is to add the core-image which includes the kernel via recopies in poky. Thats why it still boots after clearing the kernel partitions.

If one will use the kernels in the partitions, the extlinux.conf file should be changed using a bbappend. The kernel could be deleted from the rootfs but then the fallback will fail.

Is this correct?

[dwalkes]

I had the idea that the tegra-demo-distro did set all up, including the boot of A/B kernel.

It does. It's just that the kernel located in the same partition as the rootfs when using extlinux.conf support, which is the default, and only resorts to using partition content as a backup in case there are issues with the kernel located in the rootfs. So in order to do the test you performed above I think you'd need to delete the kernel out of the rootfs as well as deleting from the partition to see a boot failure for missing kernel image.

If one will use the kernels in the partitions, the extlinux.conf file should be changed using a bbappend.

I think if you want to use the kernels only in the dedicated partitions you'd set UBOOT_EXTLINUX = "0". This would mean l4t-launcher-extlinux would not be included , extlinux.conf would not be generated, and the L4TLauncher would look for the kernel image in dedicated partitions only.

[AndersGnistrup]

@dwalkes, thanks for pointing me in the right direction.

My end goal is to have a secure boot where kernel can verify the rootfs. That seems to be a preferred method AFIK.
The default setup in meta-tegra-distro includes the kernel in the rootfs so I have to define this setup in the sw-description and align that with the actual partition layout.