Reassembling a firmware from pieces

Working on firmware is always interesting. Modern x86 computers are incredibly complicated, due to the evolution of the architecture over the last 40 years, and it’s difficult to debug issues past “Well it doesn’t POST, better try something else.”

Unlike most ARM/MIPS systems, where you have a UART console or something to see output from u-boot, if you mess up the firmware on an x86, you’ll have a non-communicative brick on your hands.

Of course you can also have firmware issues on ARM/MIPS, if you manage to corrupt u-boot on SPI flash, but since u-boot is open and not proprietary it’s easy to rebuild it from source and flash it again to recover.

Not so in the PC world. UEFI is horrendously complex compared to u-boot, and Intel’s reference implementation known as TianoCore is usually “improved” by several middlemen before going into the final product.

Trammel Hudson’s excellent talk from 33c3 (slides) on Bootstrapping a slightly more secure laptop highlights the situation with UEFI quite well:

Farm to table firmware

In this case, we’ve got TianoCore from Intel, insyde adds some magic fairy dust, this goes to Compal (the ODM of this laptop), and finally Dell (who would probably also claim to add magic fairy dust). In the end there’s a lot of proprietary fairy dust floating around, and we probably wouldn’t be able to boot the laptop if we just build TianoCore from source.

TianoCore itself is open source and BSD licensed, which is why all the vendors use it. Intel manages porting TianoCore to their new platforms, and since it’s BSD licensed, it means that someone like insyde can take the working base from Intel and add their proprietary fairy dust without having to release the modified source code.

SPI flash
The start of flash contains a region called the Flash Descriptor (PDF; page 3) which is programmed at system manufacture and tells the system where different firmware components are present on flash. Think of it as a partition table for the system flash. Under normal circumstances, the flash descriptor prevents the user from reading and/or writing portions of ROM. If you try to use tools to read or write the ME regions of flash, you’ll get a error similar to this:

Error 26: The host CPU does not have read access to the target flash area. To en
able read access for this operation you must modify the descriptor settings to g
ive host access to this region.

And this makes sense. If the system allowed unrestricted write access it would be trivial for some malware to write itself into the system firmware, and then you’d have a persistent rootkit. In my opinion, blocking the ability to read portions of the firmware serves no purpose except to discourage reverse engineering attempts.

Thankfully, there is a method known as Flash Descriptor Security Override Strap which can be used to disable the flash descriptor protection.

The first step is to locate the ME_FWP pin in the circuit diagram:
Override Strap

Now that we’ve located the pin on the logical diagram, we need to find the HDA chip itself so we can see which pins we need to bridge to disable the flash descriptor lock on the ME region:
la-7731p_sda

In this case, it’s most convenient for us to short pins 5 and 9:
la-7731p_hda

You could also short pins 1 and 5, but this requires a very steady hand and small instrument. However pins 5 and 9 are connected to surface mount components located away from the chip. These components (a capacitor and a pad) are much easier to access with a wire:
la_7731p_pin5_pin9

Amazingly, this area can be accessed without completely disassembling the laptop. Just taking off the palm rest and keyboard, which is about 8 screws, is enough to access the pins. They’re right under the LVDS cable to the display. Thanks, Compal!

At this point, I have two options:

  1. Find or buy an SPI dump online and flash that
  2. Find a way to dump the firmware from the working laptop without soldering

As you read in the previous post, I only managed to find an E6320 SPI dump online, and ended up with a laptop that worked-ish. I tried for many hours, but I wasn’t able to find any free SPI dumps for the E6230 online. After seeing many forums promising to sell you the firmware for outrageous prices, I finally found one that wanted only 8 złoty ($2) to download the SPI dump for the E6230. Principles be damned, I’ve wasted enough time trying to get this to work. I paid up and downloaded the files for the 4MB and 8MB chips.

Did I really get a valid firmware for $2? Yes!

But it’s BIOS A12, the current version is A16, but none of the Dell update utilities work! I installed Windows to try the update tool from Dell, but the laptop just rebooted without updating the firmware. I tried from FreeDOS and again, the laptop would just reboot when it got to flashing. Hrmm…

So, at this point I’m going to cheat a little: I have a working E6230, but I decided when I started this that I would not touch it with a soldering iron or heat gun. If it ain’t broke, don’t fix it!

Can I get a full firmware dump from the working E6230?

Intel FPT

Intel FPT is a proprietary command-line utility created by Intel for flashing firmware files through the computer’s internal SPI flashing interface.

Alright, let’s go dump the firmware! First you need to remember to apply the Flash Descriptor Security Override Strap, or the CPU will block your read attempt to the ME region.

However, once we’ve done that, FPT will allow us to dump regions of flash to a file.

At first I tried to dump just the ME and BIOS, using the A12 firmware from above, I reflashed these regions using the FPT utility from FreeDOS. The verification of the BIOS region failed, and when I rebooted I had a brick again.

But, the FPT tool lets you dump all the regions at once! I dumped the entire flash, but now we’ve got a 12MB file and I don’t know where the split is between the 8MB and 4MB flash chips.

After more searching, I found the Intel ME System Tools, including a utility called Flash Image Tool, which allows you to import the firmware image file created by FPT.

Well, the flash component density matches what’s in the laptop, so I guess this is correct.

Flash Image Tool also lets you build a new firmware image:

Writing ROM image file “C:\Users\hmartin\Documents\MESYS\Flash Image Tool\v8.1.10.1286\Build\outimage.bin”.
Writing file “C:\Users\hmartin\Documents\MESYS\Flash Image Tool\v8.1.10.1286\Build\outimage(1).bin” (size = 8388608)
Writing file “C:\Users\hmartin\Documents\MESYS\Flash Image Tool\v8.1.10.1286\Build\outimage(2).bin” (size = 4194304)
Writing MAP file “C:\Users\hmartin\Documents\MESYS\Flash Image Tool\v8.1.10.1286\Build\outimage.map”.

Image size = 0xC00000 bytes

Interesting… outimage(1).bin is exactly 8192KB, and outimage(2).bin is exactly 4096KB. I wrote these two images to the respective chips and put them into the laptop. The moment of truth had arrived.

It boots!

When you’re trying to work through firmware issues, it’s really helpful to have the flash chip in a socket. Soldering and desoldering gets really old when you have to do it more than a couple of times.

Thoughts on the sale of firmware images

I’m against the sale of these firmware images. I realize that it takes a non-zero amount of time to get the image, but the whole experience of BIOS forums just leaves you feeling dirty. You have no way to verify before payment that the files they’re providing even work. Combine this with the fact that most websites want $10-$20 for the SPI dump, the experience leaves a bad taste in your mouth.

Is it worth it to pay? It depends entirely on your willingness to pay. Is this your only computer? Do you need it working now? How much of your own time are you willing to invest to learn about UEFI firmware? I’ve probably put 20 hours into this project, and I still don’t understand the internals of UEFI.

The wrong way to clear a BIOS password

I was browsing on eBay and ran into a listing for a Dell E6230. By now you might guess where this is leading:

Dell Latitude E6230 12.5″ i3-2350M 2.3GHz 4GB RAM Admin Pass Set AS IS READ!

s-l1600

The whole laptop, everything working perfectly, except… it has a firmware password set, and the seller doesn’t know the password. Well no bother, it’s cheap (83 EUR), I know how to flash SPI chips. Should be a piece of cake to clear this password! I also had an E6230 motherboard with a Core i5-3320M which I had found on eBay for 40 EUR, so my plan was to swap the boards, and then work on clearing the password.

These boards contain two flash chips: Winbond 64Mbit (8MB, U52) and Winbond 32Mbit (4MB, U53) located to the left of the SIM card slot.

f_0020914

The first step was to dump the SPI flash from U52 and U53 on both boards as a backup before flashing anything. This seemingly went well, but when I flashed the i5-3320M firmware dumps to the board with the i3-2350M, I had a brick.

It turns out that several of the pins to each chip are connected to the same pins in the PCH. Important pins like Data Out, Clock, and Data In are shared:

PCH SPI wiring

U52 and U53 share data out, data in, and clock lines to the PCH


I’m not sure exactly what happened, but my guess is that by leaving the flash soldered to the board, I somehow managed to get data from both chips. Alarms should have been going off in my head when my SPI reader dumped 8MB from a 4MB chip.

Not wanting to risk wrecking the i5-3320M board by desoldering the flash, I turned to the internet and discovered the terrible world of BIOS image dumps. It’s nearly impossible to find any website offering SPI dumps for free. Of course you can download the firmware update tool from the manufacturer, but those are meant to be run only on the intended hardware, and they only contain regions of the SPI flash like the UEFI firmware and Intel Management Engine.

I was able to find an SPI dump of the E6320, which is a 13.3″ laptop one generation previous to the E6230 (12.5″). As I had nothing to lose, I desoldered U52 and U53 and flashed the E6320 images to each chip. To my great surprise, the board passed POST, albeit with a warning about unsupported hardware. I was even able to enter the UEFI setup utility.

But now I am running a firmware intended for a completely different laptop. The E6320 has a QM67 (6 series) chipset, and the E6320 has a QM77 (7 series) chipset. The USB 3 ports on the side don’t work, the internal SATA port doesn’t work, and PXE booting doesn’t work. The only port that seems to work, at all, is a Mini PCIe USB port:

mini_pcie-usb30-1_zps098e038b

My next thought was to try the Dell E6230 BIOS update tool, which can run from DOS. I put FreeDOS on a USB stick in the above adapter and installed that in the WWAN Mini PCIe slot. Unfortunately for me, Dell has put checks into the update utility to check that it’s running on the correct hardware. This makes total sense, if a user downloads the wrong update for their laptop, they shouldn’t end up with a brick.

However, it didn’t suit my purpose. I wanted Dell’s BIOS update utility to ignore the fact that it was running on an “E6320” and instead flash the firmware for the E6230, the actual hardware.

Having been foiled by Dell’s checks, I decided to load up the utility in IDA Pro to see if I could bypass the check. A bit of string searching and I found the target, a jnz:
ida_jnz

Changing this check to a jz and I tried again. This time the utility didn’t complain about the machine being an E6320, but as soon as the flashing process started, the laptop shut off. So what happened? My best guess is that Management Engine shut down the platform.

Management Engine
The Intel ME has existed since the mid-2000’s, and is now deeply integrated into all of Intel’s modern x86 SKUs. The ME can provide additional functionality like a TPM (implemented in firmware), cryptographic acceleration, DRM, as well as other patented and super duper proprietary stuff. There’s a fairly comprehensive feature list available on Wikipedia.

Since Intel doesn’t actually release documents on the ME, it’s hard to come by actual information on the inner workings. It’s also why some libre people are concerned about buying newer laptops: the ME is integrated into the PCH, cannot be disabled, runs an OS with direct memory access to system RAM and has never been audited.

Trammell Hudson is currently experimenting with coreboot on the Lenovo X230, and it seems like there’s a non-zero chance that he’ll succeed in disabling the ME.

Anyway, I was on vacation earlier this year and had lots of time to kill in airports/planes/trains so I read a book about the Intel ME called “Platform Embedded Security Technology Revealed” You can download the ebook for free as a PDF or EPUB from the publisher.

Using knowledge from the above book, we can conclude that the security number of the Series 8 ME firmware must be equal to the security number of Series 7 firmware, or the ME would not allow the platform to power on.

Unfortunately, the updater managed to overwrite something important in flash before the ME cut power, because now I’m stuck with a brick again.

Stay tuned for part 2!

Allwinner H2+/H3 Ethernet with Linux 4.9-rc8

The Orange Pi PC is not a new single board computer. It’s been released for over a year now, but has mostly been stuck on a heavily patched 3.4 release kernel.

There have been ongoing efforts since the release to have support for the Allwinner H3 in the mainline kernel. In the past weeks there have been new patches released which enable support for the Ethernet MAC on the H3 (and H2+).

Unfortunately this support is not in mainline yet, and won’t make it in the upcoming 4.9 release. However, that doesn’t stop you from taking the patches and applying them against 4.9 yourself.

I wrote a script to compile the kernel from source, applying the necessary patches to the kernel and using a minimal .config file which compiles the sun8i_emac support as a module. You can download the build script from GitHub.

It does try to be somewhat smart: verifying the integrity of the downloaded files, and will bail out if there are errors in patching the source code. But, it doesn’t do toolchain dependency checking because that’s just too complicated. Since the emac support will end up in mainline soon, I doubt it’s worth the time to improve the build script. However if anyone is interested in improving it, the script is released as GPLv2.

The result? If you are patient enough to wait for the kernel to compile, you get a uImage and modules for Linux 4.9-rc8 with Ethernet support:

Allwinner H3 emac performance:

orangepi@OrangePI:~$ iperf -n 1024M -c 192.168.1.150
————————————————————
Client connecting to 192.168.1.150, TCP port 5001
TCP window size: 43.8 KByte (default)
————————————————————
[ 3] local 192.168.1.206 port 42572 connected with 192.168.1.150 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-100.6 sec 1.00 GBytes 85.4 Mbits/sec

Orange Pi Zero (Allwinner H2+):

I’ve also just received my Orange Pi Zero and confirmed that the same kernel works on the Orange Pi Zero, so you can run Linux 4.9 on the Orange Pi PC (Allwinner H3) or Orange Pi Zero (Allwinner H2+) with Ethernet support.

Allwinner H2+ emac performance:

orangepi@OrangePI:~$ iperf -n 1024M -c 192.168.1.150
————————————————————
Client connecting to 192.168.1.150, TCP port 5001
TCP window size: 43.8 KByte (default)
————————————————————
[ 3] local 172.16.4.206 port 54762 connected with 192.168.1.150 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-95.9 sec 1.00 GBytes 89.5 Mbits/sec

Download:
If you want to skip compiling the kernel yourself, I’m providing the kernel uImage and modules here.

I am using u-boot v2016.11 from denx.

Note: HDMI isn’t working on my Orange Pi PC, and since I run it headless I’m not interested in investigating why. If you’re using your Orange Pi PC with HDMI output, you may need to modify the kernel .config file to fix HDMI.