OSFV for systems without UEFI Shell + Netboot autoboot update

[SebastianCzapla]

The problem you're addressing (if any)

Currently, OSFV relies on UEFI shell for variety of keywords and tests. Systems where UEFI shell is disabled require additional work for OSFV compatibility.

Describe the solution you'd like

• Add minimal support for disable UEFI Shell, and for disable Net-autoboot

  Update uefi-shell.robot to test that expected state of UEFI Shell is there, if it is disabled in config, it must not appear in the boot options.
  USH001.001 UEFI Shell
  Do not skip this test on lack of UEFI_SHELL_SUPPORT, instead verify that no "UEFI Shell" appears in the menu constructions.
  Update network-boot.robot to test scenario where autoboot is disabled. Update config with flag that conveys expected state of autoboot, and run either PXE003.001 or new test:
  PXE008.001 Autoboot option is properly disabled
  To detect and verify no autoboot, we first verify that autoboot string does not appear in submenu construction, and then we can wait 60-120 seconds and read for expected output, to verify that we did not boot during last minute.

• Create list of all affected tests cases

• Resolve dependency issues for affected tests, e.g provide uefi shell from network

List of affected tests

USB002.001 USB keyboard detected in FW
Resolution: Use external UEFI Shell

TXE001.001 Check if platform is not fused
Resolution: Removed

STB001.001 Verify if no reboot occurs in the firmware
Resolution: Use external UEFI Shell

SBO003.001 Attempt to boot file with the correct key from Shell (firmware)
Resolution: #725

SBO004.001 Attempt to boot file without the key from Shell (firmware)
Resolution: #725

SBO005.001 Attempt to boot file with the wrong-signed key from Shell (firmware)
Resolution: #725

SBO007.001 Attempt to boot the file after restoring keys to default (firmware)
Resolution: #725

Majority of dasharo-stability/capsule-update.robot
Resolution: Out of scope for this issue, tracked separately

self-tests/terminal.robot: Execute UEFI Shell Command
Resolution: Removed

[miczyg1]

Update network-boot.robot to test scenario where autoboot is disabled. Update config with flag that conveys expected state of autoboot, and run either PXE003.001 or new test:
PXE008.001 Autoboot option is properly disabled
To detect and verify no autoboot, we first verify that autoboot string does not appear in submenu construction, and then we can wait 60-120 seconds and read for expected output, to verify that we did not boot during last minute.

Autoboot word is not present in Protectli iPXE menus. It will be just enough to read the menu completely and after 15 seconds check if any action was taken (there shouldn't;t be any). Regualr Dasahro iPXE menu has not been modified. So it shall affect only NBU tests (https://github.com/Dasharo/open-source-firmware-validation/blob/main/dasharo-compatibility/network-boot-utilities.robot). Tests from PXE suite (https://github.com/Dasharo/open-source-firmware-validation/blob/main/dasharo-compatibility/network-boot.robot) should stay intact.

USB002.001 USB keyboard detected in FW

This will be a bit complicated. To be honest, I currently have no idea how to test that without the shell. Maybe the Boot Maintenance Manager will list USB keyboard if present (https://docs.dasharo.com/dasharo-menu-docs/boot-maintenance-mgr/#console-options) which may be an indicator of working keyboard. But this needs verification.

[m-iwanicki]

Maybe replace built-in UEFI shell with external EFI shell booted via efi file or network boot? I think it should work at least in tests that don't test UEFI shell itself (or SB related ones)

[macpijan]

For testing with SB disabled, we can load shell via network.

For SB tests that specifically need SB to be enabled, we can rewrite them to not execute efi files from shell, and use boot manager instead.

[SebastianCzapla]

#697 adds bare minimum to test the new UEFI Shell defualts. Should we separate refactoring of SB tests into different PR or keep all the changes together?

[macpijan]

If we have something functional, we can merge it and followupup with further changes in another pr.

[SebastianCzapla]

@macpijan Let me know if #697 is enough for the initial PR

[macpijan]

To me it is, if someone helse has tested it

[SebastianCzapla]

@macpijan Here's an archive with runs for both tests, on rc8 and rc6 of vp3230 platform no-uefi-shell-logs.zip

[BeataZdunczyk]

Cosing as PR #697 has been merged

[macpijan]

@BeataZdunczyk This should stay open, linked MR solves it only partially. Please see the original post.