Category Archives: Firmware

Fujitsu TX1320 M3

15 Replies

As with many purchases, this one began with a thread on ServeTheHome. Given my previous work on the Fujitsu TX140 S2 motherboard, how could I refuse the offer of a barebones Fujitsu TX1320 M3 for only 29€?

Fujitsu TX1320 M3

The TX1320 M3 is slightly too tall to fit within 2U, measuring at 398x340x98mm. Those wishing to place it on a rack shelf, you will need to budget 2.5U for the unit.

My TX1320 M3 came with BIOS 1.19.0, which does not support Xeon v5 CPUs. Not knowing the iRMC S4 password, it was not possible to flash a newer BIOS via iRMC. But, the SOIC8 that contains the BIOS was easily located (on the front of the motherboard, near to the INTR header and internal USB3 port).

The BIOS update D3373-B1.ROM file provided by Fujitsu is exactly 8388608 bytes. So, updating the BIOS requires only a ch341a programmer and a SOIC8 chip-clip. Dump the current BIOS, replace the last 8MB with D3373-B1.ROM downloaded from Fujitsu, and reflash to have support for a Xeon v5 CPU:

$ sudo flashrom -p ch341a_spi -c "N25Q128..3E" -r TX1320.bin
$ dd if=D3373-B1.ROM of=TX1320.bin bs=1M seek=8
$ sudo flashrom -p ch341a_spi -c "N25Q128..3E" -w TX1320.bin

After booting Linux, it is easy to reset the iRMC S4 password using ipmitool.

iRMC S4 on the TX1320 M3 motherboard has the UART routed to an unpopulated header and has the same parameters as the TX140 S2: 38400n8.

The pin beside UART Tx is not a ground, it’s a GPIO from the Pilot III. Pin 3 of the INTR header (closest to the USB port) is GND and fits a 2.54mm prototype wire. The iRMC S4 bootlog is available in this gist.

UART access does not instantly make you god of iRMC S4. After booting, remmman is running on the uart, so without knowing a valid username and password, you can’t gain access. u-boot is built with the dhcp and tftpboot commands, so you could potentially boot a modified image from the network that would give you root access (untested).

The GPL components of iRMC S4 are available on GitHub.

The Fujitsu 250W power supply (Model: CPB09-045E, S26113-E564-V71-01) has the dimensions 93x71x187mm, which should allow for the installation of a TFX PSU (85x65x175mm) in the chassis with minimal modification. Being a Fujitsu workstation, the power supply pin-out is identical to the TX140 S2.

The SATA power connector on the motherboard is a Molex 5559 2×4 with the following pinout (top view, looking toward PCB):

GND* 5V GND 12V*
GND* 5V* GND* 12V*

The 2.5″ backplane (Fujitsu P/N: A3C40176096) has a Molex 5559 2×2 power connector with the following pinout (top view, looking toward PCB):

GND GND*
12V 12V*

Note that only the the conductors marked with * are populated in the Fujitsu wiring harness.

The 16 pin front-panel header has the same pinout as the TX140 S2:

As expected, we find two I2C EEPROMs (TSSOP8) and a Texas Instruments LM75 temperature sensor (VSSOP8) on the front-panel PCB (Fujitsu P/N: A3C40167342).

TX1320 M3 front-panel PCB

The TX1320 M3 noise profile is excellent, it is slightly audible during POST, but inaudible once booted into the OS (Linux).

The CPU cooler (V26898-B1003-V1/A3C40175673/A3C40175674) is a small dual heat pipe design with two towers with a fan (Delta AFB0712HHB) sandwiched between. The hot-swap 2.5″ drive bay has a single fan AVC DA07020B12M installed in a toolless plastic bracket. The design expects one fan per backplane, so if you install a second backplane to expand to eight 2.5″ drives, a second fan and bracket may be necessary.

lspci output:

00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers (rev 07)
00:14.0 USB controller: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller (rev 31)
00:14.2 Signal processing controller: Intel Corporation 100 Series/C230 Series Chipset Family Thermal Subsystem (rev 31)
00:16.0 Communication controller: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 (rev 31)
00:16.1 Communication controller: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #2 (rev 31)
00:17.0 SATA controller: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] (rev 31)
00:1c.0 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #5 (rev f1)
00:1c.5 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #6 (rev f1)
00:1c.6 PCI bridge: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #7 (rev f1)
00:1f.0 ISA bridge: Intel Corporation C236 Chipset LPC/eSPI Controller (rev 31)
00:1f.2 Memory controller: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller (rev 31)
00:1f.4 SMBus: Intel Corporation 100 Series/C230 Series Chipset Family SMBus (rev 31)
01:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 05)
01:00.1 Co-processor: Emulex Corporation ServerView iRMC HTI
02:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

For those looking to install 32GB unbuffered DIMMs, unfortunately there is no support for that on the TX1320 M3. The capacity is recognized, and I was able to open UEFI Setup, as well as boot the Arch Linux installer, however if the CPU addresses beyond 16GB a non-maskable interrupt (NMI) is generated and the system halts.

The TX1320 M3 is not a compelling upgrade for those who already have something like the TX140 S2: performance and power consumption are quite similar between the two generations. Only if you need more bandwidth than PCIe 2.0 offers would upgrading to the TX1320 M3 (PCIe 3.0 x8/x8/x4/x1) make sense over the TX140 S2 (PCIe 3.0 x8, PCIe 2.0 x8/x4/x1).

Power consumption:

  • 4W when powered off (iRMC powered, management NIC connected at 1000MBit)
  • 16W when idle in Linux (Xeon E3-1220 v5, 2x16GB DIMMs, 16GB boot SSD, 1xEthernet, iRMC management Ethernet)

If you do not already have a Fujitsu and are interested in a low-power server, then the TX1320 M3 (or TX1330 M2 which also uses the D3373 motherboard) is a good choice. Of course, iRMC S4 is onboard, so with a little effort you can have an Advanced license 😉

Given that the D3373 motherboard is mATX compatible, I consider it a worthwhile purchase for the chassis alone. Note that the IO shield is integrated into the chassis, so you would need to remove this with a rotary saw to install another motherboard.

D3373-B1.ROM: 8cf71990597df6561b9c7c3e2c1b7e4c4b373a7a63271ba1a93bab9f50e0903f

(The following content was added in January 2024)

If you want to expand the capacity of the system to 8 drives, you will need to purchase another backplane as the one shipped in the system only supports four 2.5″ drives.

I can confirm that the Fujitsu 8 port backplane (P/N: A3C40173252) fits mechanically with zero modifications.

The 8 port backplane can be found for the same price (29€) as most sellers are asking for the 4 port model, and simplifies the cabling. You only need one power cable and one I2C cable with the 8 port backplane, although two of each cable are provided in the TX1320. You will need an additional SATA/SAS controller though, as the D3373 has only one Mini-SAS HD connector for up to 4 devices.

You can use the original Molex 5559 power cable (2×2 positions; 2 populated) from the TX1320 with the 8 port backplane without modification; below is the pinout of the original power cable (2×3 positions; 6 populated) marked on the PCB as X40:

A3C40173252 SAS backplane power receptacle (X40)

Older revisions of the backplane also have a Micro-Fit 3.0 (Molex 430450412) populated on the PCB as X17 which offers 12V and 5V voltage outputs. Note that the below pinout is for the cable, not the X17 receptacle:

A3C40173252 X17 cable pinout

Note that you still need to provide cooling for the additional drive bays, which cannot easily be done as the plastic fan duct has no official part number (as noted by Artur in the comments).

Meraki MX84 overview

2 Replies

I became aware of the Meraki MX84 from Lathe Abusaid’s blog post about tearing down the hardware. After setting up an eBay alert and waiting, I finally won a job lot which included an MX84.

Here is a quick summary of the MX84 specs:

  • Intel Atom C2358 CPU (2C/2T, 1.74GHz)
  • 4GB DDR3 ECC RAM (H5TC4G83CFR-PBA)
  • Internal SATA port (1TB Western Digital Green)
  • External USB2.0 port
  • 13 Network interfaces (Vitesse VSC7425: 11 Gigabit Ethernet, 2 SFP)
  • 16MB SPI flash, 1GB NAND flash (Phison PS2251, USB on motherboard)
  • Fanless
  • Open frame 12V 2.5A power supply

The device runs Linux 3.18.131.

00:00.0 Host bridge: Intel Corporation Atom processor C2000 SoC Transaction Router (rev 02)
00:01.0 PCI bridge: Intel Corporation Atom processor C2000 PCIe Root Port 1 (rev 02)
00:03.0 PCI bridge: Intel Corporation Atom processor C2000 PCIe Root Port 3 (rev 02)
00:0b.0 Co-processor: Intel Corporation Atom processor C2000 QAT (rev 02)
00:0e.0 Host bridge: Intel Corporation Atom processor C2000 RAS (rev 02)
00:0f.0 IOMMU: Intel Corporation Atom processor C2000 RCEC (rev 02)
00:13.0 System peripheral: Intel Corporation Atom processor C2000 SMBus 2.0 (rev 02)
00:14.0 Ethernet controller: Intel Corporation Ethernet Connection I354 1.0 GbE Backplane (rev 03)
00:14.1 Ethernet controller: Intel Corporation Ethernet Connection I354 1.0 GbE Backplane (rev 03)
00:14.2 Ethernet controller: Intel Corporation Ethernet Connection I354 1.0 GbE Backplane (rev 03)
00:14.3 Ethernet controller: Intel Corporation Ethernet Connection I354 1.0 GbE Backplane (rev 03)
00:16.0 USB controller: Intel Corporation Atom processor C2000 USB Enhanced Host Controller (rev 02)
00:17.0 SATA controller: Intel Corporation Atom processor C2000 AHCI SATA2 Controller (rev 02)
00:18.0 SATA controller: Intel Corporation Atom processor C2000 AHCI SATA3 Controller (rev 02)
00:1f.0 ISA bridge: Intel Corporation Atom processor C2000 PCU (rev 02)
00:1f.3 SMBus: Intel Corporation Atom processor C2000 PCU SMBus (rev 02)

The MX84 uses coreboot as the bootloader (coreboot-af6fa06-dirty-Liteon_GRM1001_MFG_v4.0.0; bootlog) and the ROM has the following layout:

00000000:00010000 reserved
00010000:0070ffff bk1
00710000:00dfffff bk2
00e00000:00ffffff coreboot

The cbfs contains the following:

FMAP REGION: COREBOOT
mx84.bin: 16384 kB, bootblocksize 1024, romsize 16777216, offset 0xe00000
alignment: 64 bytes, architecture: x86

Name                           Offset     Type           Size   Comp
cmos_layout.bin                0xe00000   cmos_layout      1352 none
fallback/romstage              0xe00580   (unknown)       25820 none
fallback/ramstage              0xe06ac0   (unknown)       61965 none
fallback/payload               0xe15d40   simple elf      20349 none
config                         0xe1ad00   raw              4310 none
revision                       0xe1be00   raw               712 none
(empty)                        0xe1c100   null          1261208 none
mrc.cache                      0xf4ffc0   mrc_cache       65536 none
cpu_microcode_blob.bin         0xf60000   microcode       84992 none
(empty)                        0xf74c40   null            45912 none
fsp.bin                        0xf7ffc0   spd            389120 none
(empty)                        0xfdf000   null           134040 none

coreboot was built with an ELF payload (miles) which by default loads and jumps into the bootkernel FIT image (dts here) located at 0x10000. A secondary bootkernel exists on flash at offset 0x710000.

Let us revisit those Intel I354 interfaces. As a networking appliance, the MX84 has a lot of network interfaces.

There are 13 network interfaces on the front (Management, Internet 1 & 2, Ethernet 3-10, and 2 SFP cages) so there should be a switch inside the MX84 or we would expect to see more than four interfaces in lspci.

In this case, the switch is the VSC7425, and even if you use the 3.18.131 kernel from Meraki, you won’t have any connectivity because all four of the I354 interfaces connect directly to the VSC7425

The stock Meraki firmware uses a binary called vtss_poca_d to initialise and configure the VSC7425, which does so using a proprietary Vitesse framework (PDF).

vtss_poca_d is a static binary, so could we use it with a newer kernel such as 5.10.146 found in OpenWrt 22.03?

$ vtss_poca_d
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:0 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:1 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:2 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:3 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:0 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:1 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:2 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:3 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:0 failed: Not supported
mdio_write16: SIOCSMIIREG on eth0 phy:0 reg:1 failed: Not supported
mdio_read16: SIOCGMIIREG on eth0 phy:0 reg:2 failed: Not supported
mdio_read16: SIOCGMIIREG on eth0 phy:0 reg:3 failed: Not supported
mdio_read16: SIOCGMIIREG on eth0 phy:0 reg:2 failed: Not supported
mdio_read16: SIOCGMIIREG on eth0 phy:0 reg:3 failed: Not supported

Nope! As such, it is very unlikely this device will ever be supported by OpenWrt.

You may have noticed that the MX84 is based on the Atom C2000, a CPU which suffers from the AVR54 errata. When I first received my MX84, there was no output on UART and the power consumption was a suspiciously consistent 6W, all the hallmarks of a device dead from AVR54. There are numerous instructions for how to revive a Synology NAS with a dead Atom, but no such instructions exist for the MX84.

Fortunately for me, there was a photo from Lathe Abusaid’s blog post which provided a crucial hint. It appears that the MX84 unit photographed in their teardown includes a 100 Ohm resistor between pins 1 (LPC clock) and 9 (3.3V) of header J7.

This solution appears to have been chosen because it was the most convenient for Cisco, however note that pin 9 of J7 appears to be a GPIO output which, depending on the coreboot payload, may not always be active high. I would suggest instead soldering to pin 8 of the unpopulated SOIC8 nearby (U47), which will provide 3.3V regardless of the payload GPIO configuration.

That being said, here is a photo of the resistor fix to J7 that I just advised against doing, taken before realizing pin 9 was a GPIO

After soldering the pull-up resistor, the LPC clock is back on pin 1 of J7:

Oscilloscope output of the LPC clock on pin 1 of J7 after adding a 100Ω resistor

There is an unpopulated footprint for a Micro-USB port on the left side of the motherboard. By default the D-/D+ are not connected, as the 0 Ohm resistors are unpopulated (they are instead populated on R467/R468 connecting the Phison ps2303q to the Atom CPU). I believe this port was used during development to easily swap the USB drive connected to the SoC.

One question I had about the MX84 was: why coreboot? It seems that this design is based on Intel’s “Mohon Peak” reference platform. From the Intel customer reference board (CRB) documentation (PDF):

The embedded firmware ecosystem has developed an example boot loader solution
for the CRB that uses the FSP kit. This solution is based on the open source Coreboot
project at coreboot.org. While Intel does not endorse or support boot loader solutions
based on the Coreboot project, the example Coreboot-based boot loader provides a
good teaching model for how to integrate the Intel FSP into a complete boot loader
solution.

Now it is clear why Meraki chose to use coreboot, that is simply the bootloader reference provided by Intel for Mohon Peak. Other manufacturers who made Atom C2000 products also used coreboot (such as the VeloCloud 520-AC).

Meraki provided the coreboot source code in December 2022, after a delay of more than 12 months. The coreboot source code for the MX84 is available on GitHub.

Meraki hardware commanding the premium that it does, if you are considering buying an MX84: don’t. The VeloCloud 520-AC (C2358) and 540-AC (C2558) are available for ~$30 on eBay and have the C0 revision which doesn’t suffer from AVR54.

If you already own an MX84 and want to poke around, here is a buildroot based firmware that you flash to SPI. The firmware will boot, initialize the switch, DHCP, and start an SSH server (the root password is the device serial without hyphens). Note that it is initramfs based, so no changes are persisted.

Caveat emptor: VeloCloud devices have an issue with the igb/I354 compatibility, meaning that only the two SFP cages are functional. However, that is two more interfaces than you will get from the MX84 (zero) with any other kernel.

Fujitsu iRMC S4 License

54 Replies

A few years ago we looked at iRMC S4 on the Fujitsu TX140 S2. iRMC S4 provides typical remote management features that you would expect to find in a BMC: remote power control, sensor monitoring and alerting, hardware inventory, and boot order over-ride/selection. Some additional features like the remote KVM and remote media require a license key.

Licensed IPMI features are not new and other vendors, such as Supermicro, have had their IPMI license reverse engineered.

Fujitsu are a somewhat niche vendor when it comes to servers, and to date I am not aware that anyone has publicly reverse engineered the iRMC S4 license.

They say a picture is worth a thousand words, so we will start with a diagram

iRMC S4 license contents

An iRMC S4 license has four distinct fields

  1. Header/magic: 4 bytes (iRMC)
  2. Features to be enabled by the license (bitmask): 4 bytes
  3. Type of license (temporary or permanent): 4 bytes
  4. CRC32 of the system serial number: 4 bytes

The above data is encrypted using AES-128, and the output is base32 encoded with hyphens every 4 characters.

For example, here is an iRMC S4 license (enabling KVM and remote media) for an RX chassis with the serial number YLNS012345:

ZKAF-Z5EG-PL5G-6GFR-YEG6-CKGM-KQ

And the actual license contents:

69524d43 0300000 0ffffff05 2e4dbb51

Licensed features in iRMC S4 include:

  • Remote KVM
  • Remote media
  • eLCM

Feature bit 1 is for KVM, bit 2 remote media, and bit 3 seems to be for eLCM (eLCM appears to only be available on some models).

Installing an iRMC S4 license on a TX chassis

Back in 2014, Fujitsu changed the iRMC S4 licensing to be “node-locked”, which means that a license is tied to a specific server and cannot be transferred. The installation of a volume license is not possible after 2015-01-01 00:00:00.

iRMC S4 tracks the “Power on Hours (PoH)” of the chassis, and it appears that there is the capability to generate a temporary license which will expire after a certain number of Power on Hours is reached, probably to provide customers with time to evaluate the value proposition of purchasing iRMC licenses.

iRMC S4 time limited license

If you are reading this, then you are probably not interested in generating temporary licenses. Setting the field to 0xffffff00 for a TX chassis and 0xffffff05 for an RX chassis will result in a permanent license.

Now that we have covered the fields in an unencrypted iRMC S4 license, it will be obvious that the example license ZKAF-Z5EG-PL5G-6GFR-YEG6-CKGM-KQ is not simply the base32 encoded binary license data.

Unlike Supermicro, Fujitsu use a static HMAC message and key to create an HMAC-SHA1 hash, the first 16 bytes of which are used as the key for AES-128. The AES encrypted data is then base32 encoded and the output is the iRMC license you install via the web interface.

I will not be disclosing Fujitsu’s HMAC key and AES IV here, but suffice to say you can download and unpack the iRMC firmware from Fujitsu and find the values in /usr/local/lib/libfts_license.so.1.12.1. Thanks Fujitsu!

For anyone interested in reverse engineering the iRMC S4 license validation themselves:

  • the HMAC key and message are used in lkeyInitCipherKey in libfts_license
  • the AES IV is used in decrypt_with_license in libfts_license

libfts_license in Ghidra, showing decompiled function and hexdump

Anyone looking for a simpler solution, a proof-of-concept for python is here. Note that you need to provide the correct HMAC/AES values obtained from libfts_license, you do not need Ghidra for this, you can do it with a simple hexdump utility.

To anyone wondering, the license logic from iRMC S4 is not applicable to older iRMC platforms such as iRMC S2 or iRMC S3.

However, the license logic appears to be unchanged between iRMC S4 and S5. Hardware with iRMC S5 is too expensive to justify purchasing to verify this, but maybe someone will leave a comment as to whether the license logic described here is still applicable to iRMC S5.

Edit: An anonymous reader has written to say that the logic is unchanged for iRMC S5 ✨