Stop cloud dl

[macpijan]

The goal of this MR:

• replace: Add submodule #485
• enable ESP scanning tests on more boards (PiKVM is not realllly a requirement here)
• enable Secure Boot tests on more boards (on non-PiKVM we would need to insert stick with this image - which is now integrated into a single image, as opposed to mulltiple images before)

What should be tested:

• ESP scanning on PiKVM
• ESP scanning on non-PiKVM
• SB on PiKVM
• SB on non-PiKVM
• DTS on PiKVM
• DTS on non-PiKVM
• bios lock/earlydma/TPM/vboot on PiKVM
• bios lock/earlydma/TPM/vboot on non-PiKVM
• capsule update - at least logo persistence test
• full regression on QEMU
  • some outstanding errors in area of disks handling - won't be handled here: Refactor QEMU lib #680

Suggested order of merging:

• Odroid h4 plus regression #661
• Usb fixes integration #659
• pikvm: use package instead of submodule #665
• This MR (after rebase)

[filipleple]

@macpijan Getting instant fails here due to this issue: #628 so I've done

@@ -36,7 +36,7 @@ Get Hostname Ip
     # TODO: We do not necessarily need Internet to be reachable for the internal
     # addresses to be assigned. But if it is, the local IPs are definitely
     # already in place.
-    Wait Until Keyword Succeeds    5x    1s
+    Wait Until Keyword Succeeds    5x    10s
     ...    Check Internet Connection On Linux
     ${out_hostname}=    Execute Command In Terminal    hostname -I
     Should Not Contain    ${out_hostname}    link is not ready
     

Now, despite doing

git submodule update --init --checkout
virtualenv -p $(which python3) robot-venv
source robot-venv/bin/activate
pip install -U -r requirements.txt

after checking out the branch, I get

ESP001.001 ESP Scan with OS-specific .efi files added :: This test... | FAIL |
Parent suite setup failed:
There are no source files matching '/home/flewinski/workspace/vp6650-test/open-source-firmware-validation/osfv-test-data/esp-scanning/q
emu-disk.img'

So I'm guessing osfv-test-data is not properly registered as submodule/cloned by tests which need it? After cloning it manually I got

==============================================================================
Esp-Scanning
==============================================================================
ESP001.001 ESP Scan with OS-specific .efi files added :: This test... | FAIL |
[ Ubuntu (on SSDPR-CL100-240-G2) | Windows Boot Manager | Windows Boot Manager (on SSDPR-CL100-240-G2) | SSDPR-CL100-240-G2 | Network Boot and Utilities | UEFI Shell | Setup | USB SanDisk 3.2Gen1 | Dasharo Tools Suite (on USB SanDisk 3.2Gen1) ] does not contain match for pattern 'Suse Boot Manager (on*'.
------------------------------------------------------------------------------
ESP003.001 ESP Scan ignores OSes on removable media :: This test a... ..Mounting USB Disk Image at runtime is not supported on this platform.
ESP003.001 ESP Scan ignores OSes on removable media :: This test a... | SKIP |
Image not marked as required, skipping test case.
------------------------------------------------------------------------------
ESP004.001 ESP Scan does not create duplicate entries :: This test... | PASS |
------------------------------------------------------------------------------
ESP005.001 ESP Scan detects Dasharo Tools Suite :: This test aims ... ..Mounting USB Disk Image at runtime is not supported on this platform.
Image marked as required. Make sure that USB drive with image: /home/flewinski/workspace/vp6650-test/open-source-firmware-validation/osfv-test-data/dts/dts-base-image-v2.1.3.wic is already prepared and connected to the DUT.
ESP005.001 ESP Scan detects Dasharo Tools Suite :: This test aims ... | PASS |
------------------------------------------------------------------------------
ESP006.001 ESP Scan does not find non-block boot devices :: This t... | PASS |
------------------------------------------------------------------------------
ESP002.001 ESP Scan after deleting additional .efi files :: This t... | PASS |
------------------------------------------------------------------------------
Since the asset has been checkout manually by you prior running this script, it will NOT be checked in automatically. Please return the device when work is finished.
Esp-Scanning                                                          | FAIL |
6 tests, 4 passed, 1 failed, 1 skipped
==============================================================================

Some tests pass, but after looking at the log more closely, seems like the test doesn't manage to add any OSes to boot menu:

Arguments: [ ['Ubuntu (on SSDPR-CL100-240-G2)',
 'Windows Boot Manager',
 'Windows Boot Manager (on SSDPR-CL100-240-G2)',
 'SSDPR-CL100-240-G2',
 'Network Boot and Utilities',
 'UEFI Shell',
 'Setup',
 'USB SanDisk 3.2Gen1',
 'Dasharo Tools Suite (on USB SanDisk 3.2Gen1)'] | 'Suse Boot Manager (on*' ]	
13:16:45.365	FAIL	[ Ubuntu (on SSDPR-CL100-240-G2) | Windows Boot Manager | Windows Boot Manager (on SSDPR-CL100-240-G2) | SSDPR-CL100-240-G2 | Network Boot and Utilities | UEFI Shell | Setup | USB SanDisk 3.2Gen1 | Dasharo Tools Suite (on USB SanDisk 3.2Gen1) ] does not contain match for pattern 'Suse Boot Manager (on*'.

So the non-PiKVM approach might require some extra work

[macpijan]

There is another serious problem with this MR:

Git LFS has been disabled on the organization Dasharo because you’ve exceeded your data plan by at least 150%. Please purchase additional data packs to cover your bandwidth and storage usage:

  https://github.com/organizations/Dasharo/billing/data/upgrade

Current usage as of 16 Jan 2025 12:27AM UTC:

  Bandwidth: 0.57 GB / 1 GB (56%)
  Storage: 1.6 GB / 1 GB (160%)

So gettint these test files from github is not a good idea 😞

[macpijan]

I have resullts on non-piKVM yesterday:

Odroid ESP scanning:

Watch "logs/odroid-h4-plus/2025_01_16_00_09_38/dasharo-compatibility/esp-scanning.robot_debug.log" to monitor the progress of the test
==============================================================================
Esp-Scanning                                                                  
==============================================================================
ESP001.001 ESP Scan with OS-specific .efi files added :: This test... | PASS |
------------------------------------------------------------------------------
ESP003.001 ESP Scan ignores OSes on removable media :: This test a... ..Mounting USB Disk Image at runtime is not supported on this platform.
ESP003.001 ESP Scan ignores OSes on removable media :: This test a... | SKIP |
Image not marked as required, skipping test case.
------------------------------------------------------------------------------
ESP004.001 ESP Scan does not create duplicate entries :: This test... | PASS |
------------------------------------------------------------------------------
ESP005.001 ESP Scan detects Dasharo Tools Suite :: This test aims ... ..Mounting USB Disk Image at runtime is not supported on this platform.
Image marked as required. Make sure that USB drive with image: /home/macpijan/projects/github/dasharo/open-source-firmware-validation/osfv-test-data/dts/dts-base-image-v2.1.3.wic is alredy prepared and connected to the DUT.
ESP005.001 ESP Scan detects Dasharo Tools Suite :: This test aims ... | PASS |
------------------------------------------------------------------------------
ESP006.001 ESP Scan does not find non-block boot devices :: This t... | PASS |
------------------------------------------------------------------------------
ESP002.001 ESP Scan after deleting additional .efi files :: This t... | PASS |
------------------------------------------------------------------------------
Since the asset has been checkout manually by you prior running this script, it will NOT be checked in automatically. Please return the device when work is finished.
Esp-Scanning                                                          | PASS |
6 tests, 5 passed, 0 failed, 1 skipped

So the logic is probably fine, but the GH LFS is a big miss.

[filipleple]

Hmm, I've got the same result again, using the repo mirror from cloud instead of gh lvs. Not sure what makes the VP6650 a corner case.

==============================================================================
ESP001.001 ESP Scan with OS-specific .efi files added :: This test... | FAIL |
[ Ubuntu (on SSDPR-CL100-240-G2) | Windows Boot Manager | Windows Boot Manager (on SSDPR-CL100-240-G2) | SSDPR-CL100-240-G2 | Network Boot and Utilities | UEFI Shell | Setup ] does not contain match for pattern 'Suse Boot Manager (on*'.
------------------------------------------------------------------------------
ESP003.001 ESP Scan ignores OSes on removable media :: This test a... ..Mounting USB Disk Image at runtime is not supported on this platform.
ESP003.001 ESP Scan ignores OSes on removable media :: This test a... | SKIP |
Image not marked as required, skipping test case.
------------------------------------------------------------------------------
ESP004.001 ESP Scan does not create duplicate entries :: This test... | PASS |
------------------------------------------------------------------------------

Here's the full log