Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot reboot after successful unlocking since BIOS update #404

Open
michalwoz opened this issue Jul 9, 2022 · 13 comments
Open

Cannot reboot after successful unlocking since BIOS update #404

michalwoz opened this issue Jul 9, 2022 · 13 comments

Comments

@michalwoz
Copy link

I've update BIOS in Dell Latitude 5421 from 2021 year and since than Sedutil don't reboot after successful unlock. It just stuck on "Starting OS"
I've downgraded BIOS, but it didn't solve the issue.

"reboot" command from rescue disk also doesn't work properly. I managed to edit syslinux.cfg and remove acpi=off noapic
Now I can reboot when booted from rescue disk, but still cannot reboot when booted from shadow partition.

So I've mounted shadow partition when booted from rescue disk, edited syslinux.cfg removed acpi=off noapic on shadow partition, hoping that shadow partition will be able to reboot as rescue disk did. But the change of syslinux.cfg is not persistent. acpi=off noapic reappears in file after reboot. How can I solve that?

There is @ChubbyAnt fork with no acpi=off noapic in both syslinux.cfg so I tried to load his PBAimage to shadow partition but got 'NOT AUTHORISED'. @ChubbyAnt's rescue disk reboots successfully.

I've also searched any setting in BIOS before and after update which can change ACPI behawior but with no success.

@michalwoz
Copy link
Author

After 12+hour fight I fixed it. Finally I've recreated UEFI64.img with syslinux.cfg without acpi=off noapic part. I loaded this new img with --loadPBAimage . Accidentally I've broken Windows EFI partition so I need to fix it too.

@lapause
Copy link

lapause commented Jul 16, 2022

@michalwoz I'm stuck in the same situation after upgrading the BIOS of my Precision 7560 to v1.13. Could you detail what steps you followed to properly update the UEFI64.img file?

@michalwoz
Copy link
Author

@michalwoz I'm stuck in the same situation after upgrading the BIOS of my Precision 7560 to v1.13. Could you detail what steps you followed to properly update the UEFI64.img file?

I've manually executed the following lines of code:

# make an image file
dd if=/dev/zero of=${BUILDIMG} bs=1M count=32
(echo "n";echo "";echo "";echo "";echo "ef00";echo w;echo Y) | gdisk ${BUILDIMG}
LOOPDEV=`sudo losetup --show -f -o 1048576 ${BUILDIMG}`
sudo mkfs.vfat $LOOPDEV -n ${BUILDTYPE}
sudo mkdir image
sudo mount $LOOPDEV image
sudo chmod 777 image
sudo mkdir -p image/EFI/boot
sudo cp ../scratch/${SYSLINUX}/efi64/efi/syslinux.efi image/EFI/boot/bootx64.efi
sudo cp ../scratch/${SYSLINUX}/efi64/com32/elflink/ldlinux/ldlinux.e64 image/EFI/boot/
sudo cp ../scratch/buildroot/64bit/images/bzImage image/EFI/boot/
sudo cp ../scratch/buildroot/64bit/images/rootfs.cpio.xz image/EFI/boot/
sudo cp ../buildroot/syslinux.cfg image/EFI/boot/
sudo umount image
sudo losetup -d $LOOPDEV

I've to do it manually, because there is something wrong with git repo used in getresources script. So instead of using getresources script I extracted resources from latest .img file. As a result the only difference in original image and my image is deletion of acpi=off noapic from syslinux.cfg file.

@sedut-pank
Copy link

Help with a similar problem, please
#418

@nspo
Copy link

nspo commented Aug 8, 2023

For both an alternative way to modify the image files and links to the modified versions based on 1.20.0, see here: #415 (comment)

@opekar
Copy link

opekar commented Sep 8, 2023

had the same problem after updating dell bios choosed the same way as @michalwoz

#take official UEFI image and change the kernel params remove "acpi=off noapic" 
root@vo:~ gunzip UEFI64-1.20.0.img.gz
root@vo:~ kpartx -l UEFI64-1.20.0.img 
loop0p1 : 0 63455 /dev/loop0 2048
root@vo:~ kpartx -a -v UEFI64-1.20.0.img 
add map loop0p1 (253:0): 0 63455 linear 7:0 2048
root@vo:~ mount /dev/mapper/loop0p1 /mnt/uefi2/
#remove "acpi=off noapic"  from the kernel params
root@vo:~ vi /mnt/uefi2/EFI/boot/syslinux.cfg
root@vo:~ umount /mnt/uefi2 
root@vo:~ kpartx -d UEFI64-1.20.0.img 

put it on the flash,then boot the rescue image from another flash,
mount the flash with UEFI64-1.20.0.img to mnt (e.g. mount /dev/sdb1 /mnt)
sedutil-cli --loadpbaimage /mnt/UEFI64-1.20.0.img /dev/nvme0

and reboot

@elevenfive
Copy link

Workaround that solved the issue for me: downgrade to 1.15.1.01.

@pedrib
Copy link

pedrib commented Nov 16, 2023

Had sedutil working fine on my old laptop, got a new laptop with an Intel 13th gen CPU and would get stuck at "Starting OS" after entering the correct password.

Got hints of what was going on here: #415 (comment)
Like the poster, removing acpi=off noapic didn't solve my problem. Downgrading to 1.15 or using @ChubbyAnt's fork didn't help, nor using any other forks.

The final solution was to upgrade the kernel bzImage to the latest 5.15 LTS release AND remove acpi=off noapic from the syslinux.cfg file.

Booted into sysrescue CD and followed the instructions below.

1- Downloaded latest (1.20) sedutil_LINUX.tgz and UEFI64.img.gz from https://github.com/Drive-Trust-Alliance/sedutil/wiki/Executable-Distributions

2- Mounted UEFI image according to instructions here: #404 (comment)

gunzip UEFI64.img.gz
kpartx -l UEFI64.img 
kpartx -a -v UEFI64.img
mount /dev/mapper/loop1p1 /mnt/uefi/

3- Downloaded latest 5.15 LTS release from kernel.org (5.15.138 at the time) and unzipped it to the folder

4- Copied old config into the LTS tree and created a new config

cp images/buildroot/64bit/kernel.config linux-5.15.138/.config
cd linux-5.15.138
make oldconfig
# keep pressing enter to choose all defaults until config is done
make -j`nproc` bzImage

5- Copy newly built bzImage back to UEFI image
arch/x86/boot/bzImage /mnt/uefi/EFI/boot/bzImage

6- Edit /mnt/uefi/EFI/boot/syslinux.cfg and remove acpi=off noapic

7- Unmount modified UEFI image

umount /mnt/uefi
kpartx -d UEFI64.img

8- Verify changes have been written

sudo mount -o loop,offset=1048576,rw UEFI64.img /mnt/uefi/
ls -l /mnt/UEFI/EFI/boot
umount /mnt/uefi

(file dates should differ)

9- Write the PBA image to the disk
sedutil-cli --loadPBAimage #PASS# UEFI64.img /dev/nvme1

All is working now on a latest gen laptop. Thanks to all the hints provided by the posters in multiple issues here in GitHub.

I'm attaching here the modified kernel config for 5.15.138.
5.15.138-config.zip

@emfol
Copy link

emfol commented May 9, 2024

@pedrib I followed your directions and now it's working nicely. Thanks for sharing this!

@zygisk22
Copy link

I recently purchased a brand spanking new MSI laptop with brand spanking new chipsets. As a long time user of TCG Opal enabled SSD's and sedutil-cli I was excited to boot the official UEFI64.img PBA from https://github.com/Drive-Trust-Alliance/sedutil/wiki/Executable-Distributions. Now I've had issues in the past, mainly USB adapter support; and also cold reboots after unlocking the internal drive thereby power cycling and relocking it which was solved simply by editing the syslinux.cfg and removing "acpi=off" and "noapic". I'm also quite familiar with many of the sedutil forks out there and all the small features they each can bring whether it's different hashing algorithms, support for pyrite and other variations, support for Opal over NVME to USB adapters, Windows NVME support, and more.

However, this time was quite a challenge because initially when booting the PBA image I got stuck at

Loading bzImage... ok
Loading rootfs.cpio.xz...ok

and that's all she wrote. There I was stuck. I tried endless different forks and syslinux.cfg edits, in addition to basically changing every bios option I had until eventually...it booted the standard vanilla unedited Drive-Trust-Alliance UEFI64 PBA as linked above. There I was, just had to put in the password, except, as I soon realized, it wasn't recognizing the internal NVME drive at all because it wasn't recognizing the NVME controller. That's because the kernel was too old and this was a brand new chipset, and I confirmed this because booting a recent linux distro with a newer kernel and running that same DTA sedutil-cli executable version did indeed recognize the ssd drive.

So after previously trying a bunch of different things, including trying to swap out the bootx64.efi and ldlinux.e64 with newer versions to get it to even boot the PBA (unsuccessfully) I realized the kernel in the UEFI64 PBA (and RESCUE64 image for that matter) needed to be updated. That would be the bzImage file.

Luckily I found some instructions here #404 (comment). Unfortunately though, easier said than done as these instructions did not work (specifically the recompiling of the kernel) due to missing packages, needing a specific processor type, and a lack of knowledge on my part. With some help from my friend who enjoys compiling random things and some elbow grease on his part we were able to compile the bzImage with a couple new kernels (from kernel.org) and even integrated them into the original vanilla UEFI64 PBA from the Executable-Distributions link above (I think it's version 1.20 but hard to tell because this Drive-Trust-Alliance repository is lacking some maintenance). The syslinux.cfg file was also edited to remove "acpi=off" and "noapic". These new linux kernel version PBA's are attached here for your downloading pleasure (unzip before using):

linux-5.15.157_UEFI64.img.zip
linux-6.6.29_UEFI64.img.zip

Success! Both worked and recognize the new chipset, and subsequently the internal PCI-E NVME ssd, and it even solved the boot issue where it would get stuck at "Loading rootfs.cpio.xz...ok" seemingly without even having to mess with the bios settings.

Now let's say you don't like this vanilla sedutil flavor, all you have to do is download one of these two EUFI64 images with the kernel of your choice, extract the bzImage from it (PeaZip and 7zip both work nicely or just mount the image and copy) and use it to replace the bzImage in your sedutil boot image variant of choice. It can be either the PBA or RESCUE image both work the same. Linux instructions on editing the boot images are already linked in the issue 404 comment linked above, but if by chance you're running windows here's an easy method to not only replace the bzImage but also edit the syslinux.cfg file on your usb boot media of choice. There are many ways to skin this cat but here's a way for you to quickly test it with the PBA image of your choice:


1 - burn your EUFI64.img version of choice to a usb (I prefer rufus portable, make sure to click "List USB Hard Drives" if you don't see your drive)

2 - start powershell as administrator

3 - run this command

Add-PartitionAccessPath -DiskNumber 1 -PartitionNumber 1 -AccessPath "Z:"

you can find the DiskNumber and PartitionNumber of your usb drive under the Disk Management utility, or just search the web on how to list partitions in Powershell. AccessPath is the drive letter where you will mount this EFI partition, make sure it is free.

4 - launch a file explorer as Administrator (i used total commander) since you will require Admin priveleges to browse this drive and at that point you can manipulate the file system and edit files normally. Switch out the bzImage, and edit the syslinux.cfg if you feel inclined.

5 - unmount the partition before removing the drive as a best practice

Remove-PartitionAccessPath -DiskNumber 1 -PartitionNumber 1 -AccessPath "Z:"

6 - safely remove the usb drive and boot your computer with it


If all went according to plan and it works and you want to write the PBA to your drive's shadow MBR you can then go about 1) downloading and mounting the original EFI boot img file and editing it directly or 2) just rip/image the partition you burned to the usb drive or 3) recompile the entire PBA from scratch. Then with that raw EFI image you can use the sedutil-cli --loadPBAimage command to write it to the Opal drive's read-only shadow MBR partition.

Alternatively just keep booting the PBA/RESCUE image and unlocking the drive with your external usb drive for some obfuscation and plausible deniability purposes.

@elevenfive
Copy link

Note to any readers of the above suggestion: in a post-xzutils world it is not a good idea to be running random binaries from random people w/o published diff's. If the above author would care to publish some diffs that would be a lot more helpful.

@gohrner
Copy link

gohrner commented Jun 25, 2024

@elevenfive: This only helps you if

  1. you actually check what the diff contains
  2. you actually check that the diff really is the diff between the before and after state
  3. the "before" state was "clean" / trustworthy in the first place - all of these binaries in this project are pretty "niche" as far as I'm aware, and are probably not thoroughly checked by too many people (if at all)

Anything else would be imagined security, which may actually be worse than being aware of the risks (which can be critical, obviously).

@zygisk22
Copy link

As new computer chipsets are released the chances that the sedutil PBA will malfunction increase as this minimal bootable linux image becomes outdated and requires new kernels for compatibility and modified settings in order to deal with unique hardware environments and BIOS configurations. I just got a new ASUS Expertbook P5 Intel Core Ultra 7 laptop and despite compiling even the newest linux kernels for the UEFI64.img PBA I was back to square one with the system freezing right after "Loading rootfs.cpio.xz...ok" appears.

So how did I troubleshoot and eventually solve this issue? I will share my methodology in the hopes of helping others as I've come to understood that these errors can be unique to each system. First I tried every kernel version of the PBA I have and they all stalled and about a minute after also reset the computer. These images were working successfully on many other desktops and notebooks which I tested. So after experimenting with a few standard append flags in the syslinux.cfg file I eventually went into my BIOS and disabled EVERYTHING . It booted and I was able to successfully unlock my drive after I re-enabled the nvme drives in the BIOS. Then systematically I started to re-enable other settings (except secure boot that needs to be disabled for the PBA to boot)...until eventually everything was back to default and it still worked! However when I fully reload the BIOS defaults it stops working. How is that possible? Turns out that certain settings when disabled and re-enabled change hidden advanced settings in the BIOS. Luckily I was able to pinpoint those exact settings because when you save the BIOS settings it lists all the changes, and even lets you click on them enabling access to the hidden Advanced menus in the BIOS. This happens to be the case with ASUS, each manufacturer's BIOS is different.

So turns out that disabling and enabling the Intel Trusted Execution and VT-d (including saving and rebooting the bios in between) leaves the hidden CPU setting "X2APIC" disabled, whereas by default it is enabled. In my case this was the culprit (although there were a few other seemingly insignificant hidden settings that also weren't restored to default). At this point I could leave it disabled but considered what if I prefer to keep my BIOS in its fully default state and so I continued on a Google/ChatGPT journey to research how to deal with this on the syslinux/kernel layer. I reloaded my BIOS defaults, and was back to the PBA boot sequence freezing.

There were endless append flags which I tried modifying in the syslinux.cfg such as those related to the BIOS settings I changed, including "nox2apic" which didn't help. There were countless debugging flags which also didn't show any errors. I even streamlined the testing process by removing the "noescape 1" and changing the "prompt" value to 1 which gave me a "boot:" console to quickly test out different flags without remounting and rewriting the image each time. All I had to type was "DTA [flags]" inputting which ever flags I wanted and then quickly resetting the computer when it stalled.

I tried a lot of flags along with a newer kernel, and was about to give up, until finally one simple flag saved the day: "nosmp". It forces the kernel to use only a single cpu core, bypassing all the issues with multi-core functionality. Luckily we only need a single core for the sedutil PBA to work as it's simply unlocking the Opal drive, and once it warm reboots this doesn't affect whichever operating system is loaded at that point. Here's the full contents of the syslinux.cfg which I'm successfully using now.


default DTA
prompt 0
noescape 1
label DTA
kernel bzImage
initrd rootfs.cpio.xz
append loglevel=0 nosmp libata.allow_tpm=1


And here is this syslinux.cfg file nicely compiled with linux kernel 6.13.3 for those who might find it convenient:

linux-6.13.3_nosmp_UEFI64.zip

Yes I agree with @elevenfive it's better to compile everything yourself, and I have shared the changes I've made for this purpose. However this turnkey download option is still available for those who determine that the time/security tradeoff is worth running a pre-compiled package, akin to pretty much 99% of the software people are using today. Each user should consider the potential risks, threat vectors, and sensitivity of their own situation.

Please note, newer kernels don't necessarily mean they will work just because an older one did. On one of the laptops i tested a 5.x kernel was successfully booting whereas a 6.x was stalling. Each system is unique and might require trial and error with kernel versions, bios settings, and append flags.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants