Category Archives: Networking

Meraki MX85 hardware overview

Leave a reply

The Meraki MX85 SD-WAN appliance (codename “Box Wine”) is the replacement to the Meraki MX84 and offers 4 WAN uplink ports (2 SFP, 2 Gigabit Ethernet, 1 w/PoE), 10 LAN ports (8 Gigabit Ethernet, 2 SFP), a dedicated Gigabit Ethernet port for management, and a USB 3.0 port for external cellular modems¹.

Inside the Meraki MX85

Here is a summary of the MX85 specs:

  • NXP LayerScape LS1046A (ARM A72, 4 cores @ 1.8GHz)
  • 8GB DDR4 RAM (Samsung K4AAG165WA-BCWE x4, soldered)
  • 16GB of EMMC flash (SanDisk SDINBDA6-16G)
  • Winbond W25Q64JVSIQ (x2)
  • Aikido/Cisco TAM hardware root-of-trust (Microchip SmartFusion2 M2S010)
  • Qualcomm QCA8337-AL3C 7-port Gigabit Ethernet Switch (x2, PDF datasheet)
  • Qualcomm QCA8334-AL3C 4-port Gigabit Ethernet Switch (PDF datasheet)
  • Atheros AR8033-AL1A Gigabit Ethernet PHY (dedicated management port)
  • Microchip PD69104B1 PSE controller (PoE WAN port)
  • UMEC UP1501D-54 150W power supply

Meraki tries to be the Apple of SMB networking, and frequently uses premium materials like aluminum in their product designs (MS220, MS320, MS225, MS350, MX84). This is a bit silly for something that sits in a rack, but it is the brand image they were trying to cultivate.

The MX85 does not appear to use any aluminum in the chassis. Like the budget-oriented MS120 series, the entire MX85 chassis is made of steel. Meraki marketing will tell you this was for better cooling and is definitely not related to any cost reduction.

Meraki engineers even included thermal pads and metal spacers on top of the SFP ports (and below the PCB) to dissipate heat through the chassis. You could be forgiven for assuming they are SFP+ ports (they are not) with so much attention given to heat dissipation.

All for a device which consumes less power at idle than the (also) passively-cooled MX84, and no longer includes a spinning hard drive.

Unlike previous products, Meraki use glue to secure the front panel ribbon cable

The UART header is J3 on the MX85 and follows the standard Meraki UART pinout (1: VCC, 2: Tx, 3: Rx, 4: GND) at 3.3V and 115200 baud.

Note: R6 and R7 are 0 ohm resistors which (dis)connect Tx and Rx lines of the SoC to the UART header. R6/R7 are not populated by default. You must populate them, or bridge the pads, for the UART header to function.

The U-Boot release on the MX85 is 2018.09julia-spl-boxwine and, like all other recent Meraki products, it does not allow interrupting boot.

U-Boot SPL 2018.09julia-spl-boxwine (Mar 17 2021 - 20:02:01 +0000)
Initializing DDR....using SPD
DDR clock (MCLK cycle 952 ps) is slower than DIMM(s) (tCKmax 750 ps) can support.
Trying to boot from BOOTROM


U-Boot 2018.09julia-spl-boxwine (Mar 17 2021 - 20:02:01 +0000)

SoC:  LS1046AE Rev1.0 (0x87070010)
Clock Configuration:
       CPU0(A72):1800 MHz  CPU1(A72):1800 MHz  CPU2(A72):1800 MHz  
       CPU3(A72):1800 MHz  
       Bus:      700  MHz  DDR:      2100 MT/s  FMAN:     800  MHz
Reset Configuration Word (RCW):
       00000000: 0e150012 10000000 00000000 00000000
       00000010: 33330000 00b00012 40000000 c1000000
       00000020: 00000000 00000000 00000000 00018ffc
       00000030: 20004504 01003000 00000096 00000001
Model: LS1046A RDB Board
Board: LS1046ARDB, boot from Invalid setting of SW5
CPLD:  V0.0
PCBA:  V0.0
SERDES Reference Clocks:
SD1_CLK1 = 100.00MHZ, SD1_CLK2 = 100.00MHZ
I2C:   ready
DRAM:  Detected UDIMM Fixed DDR on board
DDR clock (MCLK cycle 952 ps) is slower than DIMM(s) (tCKmax 750 ps) can support.
7.9 GiB (DDR4, 64-bit, CL=15, ECC off)
SEC0: RNG instantiated
PPA Firmware: Version LSDK-18.09
GPIO:	initialized
setting up RGB LED controller lp5562....
Using SERDES1 Protocol: 13107 (0x3333)
Using SERDES2 Protocol: 0 (0x0)
SERDES2[PRTCL] = 0x0 is not valid
NAND:  0 MiB
MMC:   FSL_SDHC: 0
EEPROM: meraki_MX85 600-102010
In:    serial
Out:   serial
Err:   serial
Net:   Invalid SerDes protocol 0x3333 for LS1046ARDB
Fman1: Uploading microcode version 108.4.9
Could not get PHY for MDIO2: addr 8
Failed to connect
Could not get PHY for MDIO2: addr 9
Failed to connect
Could not get PHY for MDIO1: addr 9
Failed to connect
PCIe0: pcie@3400000 disabled
PCIe1: pcie@3500000 disabled
PCIe2: pcie@3600000 disabled
FM1@DTSEC3, FM1@DTSEC4, FM1@DTSEC5 [PRIME], FM1@DTSEC6, FM1@DTSEC9, FM1@DTSEC10

As we can see from the above ECC off output, the MX85 is using non-ECC RAM. This is a downgrade from the MX84 which did use ECC memory.

The MX85 also contains the Cisco TAM, implemented using a SmartFusion2 M2S010. The TAM is used for secure boot.

## Starting application at 0x82120000 ...
bootselect
## Application terminated, rc = 0x0
## Starting application at 0x82120000 ...

----Security Versions----
SecureBoot:  R6.3.66-f6737c7-20200623
SB Core:     F01257R21.038ae8d0b2020-05-15
Microloader: MK0007R01.0105062020
SF: Detected SPI Generic with page size 256 Bytes, erase size 4 KiB, total 16 MiB

----SecureBoot Registers----
system_invalid:            0
boot_check_count_error:    0
boot_done:                 1
boot_ok:                   1
boot_check_count_golden:   0
boot_check_count_upgrade:  2
boot_status_golden:        0
boot_status_upgrade:       1
first_bootloader:          1

----Upgrade----
boot_error:                0
boot_check_count_error_vc: 0
boot_check_count_error:    0
boot_timeout_vc:           0
boot_timeout:              0
boot_cs_good:              1
boot_config_error:         0
boot_version_error:        0
boot_config_error_code:    0
boot_error_code:           0
boot_cs_good:              1
boot_version_error:        0
boot1_cs_key_type:         1
boot1_cs_return_code:      0
boot1_cs_key_index:        5
boot2_cs_return_code:      0
boot2_cs_key_index:        5
boot2_cs_key_type:         1

----Other Registers----
fpga_version:      0090

Reading whitelist from TAM
whitelist.bin: 740 bytes

Converting whitelist to signature fdt
BOX-WINE_LDWM-rel
wired-arm64-AP-SECP384R1_1-rel
wired-arm64-OD-SECP384R1_1-rel
wired-arm64-RT-SECP384R1_1-rel
wrote 558 bytes to 0000000082330000
## Application terminated, rc = 0x0
** File not found part.new **
87760567 bytes read in 4176 ms (20 MiB/s)
## Loading kernel from FIT Image at a0000000 ...
   Using 'conf@3' configuration
   Verifying Hash Integrity ... sha384,secp384r1:wired-arm64-RT-SECP384R1_1-rel+ OK
   Trying 'kernel@1' kernel subimage
     Description:  Linux kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0xa000012c
     Data Size:    10563592 Bytes = 10.1 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x80080000
     Entry Point:  0x80080000
     Hash algo:    sha1
     Hash value:   186b252be8c267ec7b20b072de98fe3d51c93c7f
   Verifying Hash Integrity ... sha1+ OK
## Loading ramdisk from FIT Image at a0000000 ...
   Using 'conf@3' configuration
   Verifying Hash Integrity ... sha384,secp384r1:wired-arm64-RT-SECP384R1_1-rel+ OK
   Trying 'ramdisk@1' ramdisk subimage
     Description:  meraki-image
     Type:         RAMDisk Image
     Compression:  gzip compressed
     Data Start:   0xa0a13224
     Data Size:    76964193 Bytes = 73.4 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: unavailable
     Entry Point:  unavailable
     Hash algo:    sha1
     Hash value:   a1f027fbf5acbf81befdb6ce746fee76adf132d5
   Verifying Hash Integrity ... sha1+ OK
## Loading fdt from FIT Image at a0000000 ...
   Using 'conf@3' configuration
   Verifying Hash Integrity ... sha384,secp384r1:wired-arm64-RT-SECP384R1_1-rel+ OK
   Trying 'fdt@3' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0xa538fb0c
     Data Size:    46124 Bytes = 45 KiB
     Architecture: AArch64
     Load Address: 0x90000000
     Hash algo:    sha1
     Hash value:   dd869c604072a7e29f37cc6cb4e1c9c398a46295
   Verifying Hash Integrity ... sha1+ OK
   Loading fdt from 0xa538fb0c to 0x90000000
   Booting using the fdt blob at 0x90000000
   Loading Kernel Image ... OK
   Using Device Tree in place at 0000000090000000, end 000000009001e42b
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
fdt_update_ethernet_dt: Invalid SerDes prtcl 0x3333 for LS1046ARDB
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
WARNING failed to get smmu node: FDT_ERR_NOTFOUND
*** din = 0x0000000000000000

All ahead full! Goodbye!

All head full! Screw all attempts to boot any other software on this device! Let the LIC-MX85-SEC-3Y embrace your wallet!

To anyone still wondering: no, there will never be OpenWrt support for this device.

Idle power consumption: ~15W

The power supply in the MX85 is the same model (UMEC UP1501D-54) found in the MS220-8P and the MS120-8FP. It is rated for 2.7A at +56VDC

UMEC UP1501D-54 label

Model Codename Part number
MX85 Box Wine 600-102010

The codename of the MX85 might be “wines” there are multiple references to both in the bootloader and firmware.

¹: USB modems with MX/Z series devices running firmware MX 18 or newer will be limited to best effort support and will not be receiving any future firmware fixes or improvements. Meraki documentation

It would seem that Meraki prefers their customers purchase an MG41 or MG51 than plug in their own USB LTE modem. Better margins and less to support, win-win!

The GPL source code for the MX85 was requested from Meraki in May 2024. At the time of writing Meraki has not provided any of the requested source code.

Meraki MS420 hardware overview

Leave a reply

The Meraki MS420 series switches (codename “Fatboy”) offer 24 or 48 ports of 10G SFP+, with a dedicated gigabit Ethernet port for remote management.

The MS420 series does not support dedicated stacking capabilities, although you can always connect multiple switches together via SFP+.

The MS420 was discontinued in 2016, and is too old to support secure boot.

Here is a quick summary of the MS420 specs:

  • Broadcom BCM56840 family “Trident+” ASIC (Product Brief PDF). The BCM56846 is used for both 24 and 48 port models.
  • BCM84754 SFI-to-XFI PHY
  • Freescale P2020E (PowerPC) dual-core management CPU @ 1GHz
  • 128MB NAND (Hynix H27U1G8F2BTR-BC)
  • 2048MB DDR3 ECC RAM (soldered)

Like the PowerPC-based Meraki MX60 and MX80, the MS420-series uses Kernel 3.4.113.

Similar to other Meraki Broadcom-based switches, the MS420 series implements the packet engine in userspace, using the linux_kernel_bde and linux_user_bde kernel modules to interface with the ASIC. In the Meraki firmware, the packet engine is a component of the userspace click daemon, which loads the bcm_click shared object during click router initialisation.

The UART header is CONN4 on the MS420 and follows the standard Meraki UART pinout (1: VCC, 2: Tx, 3: Rx, 4: GND) at 115200 baud.

The u-boot release on the MS420 is 2013.10 and allows interrupting the autoboot with the magic string xyzzy, as found on other older Meraki products before they disabled the boot delay. Therefore, you can interrupt the boot process, although you have only a few seconds to do so:

NAND boot... transfering control to secondary loader
Starting secondary loader ...
Booting U-boot
transfering control


U-Boot 2013.10-g9a6f165 (Mar 11 2016 - 20:09:33)

CPU0:  P2020E, Version: 2.1, (0x80ea0021)
Core:  e500, Version: 5.1, (0x80211051)
Clock Configuration:
       CPU0:1000 MHz, CPU1:1000 MHz, 
       CCB:333.333 MHz,
       DDR:333.333 MHz (666.667 MT/s data rate) (Asynchronous), LBC:41.667 MHz
L1:    D-cache 32 KiB enabled
       I-cache 32 KiB enabled
Board: Fatboy
I2C:   ready
DRAM:  2 GiB (DDR3, 64-bit, CL=6, ECC on)
L2:    512 KiB enabled
NAND:  128 MiB
*** Warning - bad CRC, using default environment

PCIe1: Root Complex, x1, regs @ 0xff70a000
  01:00.0     - 14e4:b846 - Network controller
PCIe1: Bus 00 - 01
PCIe2: Root Complex, no link, regs @ 0xff709000
PCIe2: Bus 02 - 02
In:    serial
Out:   serial
Err:   serial
init GPIO
init REAR FAN I2C to output
register sysled device
Set Fan in Full Speed
Net:   eTSEC1 [PRIME]
autoboot in 3 seconds
LOADER=>

Once you have gained access to the u-boot console it is possible to tftpboot your own payload.

The P2020E boots directly off NAND, without an intermediate “bootkernel” on SPI flash (as is the case on the MS220, MS210/225, and MS350).

Booting is not as straightforward as just creating a FIT image. Similar to the MX80, Meraki have created a custom image format which includes a header with additional data used to validate the integrity of the image.

The layout of the image header for the MS420 is as follows:

Header field Data type (value)
SHA1_MAGIC uint32 (0x8e73ed8a)
DATA_OFFSET int32 (0x0000400)
DATA_LEN int32
SHA1SUM char[20] 
0000:00:00.0 PCI bridge: Freescale Semiconductor Inc P2020E (rev 21)
0001:02:00.0 PCI bridge: Freescale Semiconductor Inc P2020E (rev 21)
0001:03:00.0 Ethernet controller: Broadcom Corporation Device b846 (rev 02)

The switch contains a discrete I2C RTC (NXP PCF8563), however there is no battery present so the RTC does not retain the time across power cycles.

rtc-pcf8563 2-0051: low voltage detected, date/time is not reliable.
rtc-pcf8563 2-0051: setting system clock to 2010-03-21 03:00:00 UTC (1269140400)

The four 40mm system fans in the MS420 are controlled by an onsemi ADT7473 (PDF datasheet). It appears that only the internal temperature sensor of the ADT7473 is present (for anyone else wondering, digging into the kernel module reveals that temp1 corresponds to Remote 1 Temperature, temp2 to Local Temperature, and temp3 to Remote 2 Temperature). The Meraki firmware does a respectable job of managing fan speed; after booting and settling, the fans are not quite the 1U screamer as you might expect from such a switch. They are still audible, but it is tolerable and being near the switch does not require an investment in hearing protection.

The MS420 fans have a Meraki part number: FAN-MS420-R (P/N 680-29010). These are identical to the Cisco FAN-T1, which can be purchased for considerably less than the Meraki branded part.

The MS420 accepts two hot-swap power supplies (model PWR-MS420-400AC-R, P/N 640-29010), which in my units are Compuware model CPR-4011-4M1 with 12V/33A and 5Vsb/3A output (combined power 400W) and are 80+ Gold certified:

MS420-24 idle power consumption: ~85W (single PSU)
MS420-48 idle power consumption: ~100W (single PSU)

The GPL source code for the MS420 was requested from Meraki in July 2023 and at the time of writing Meraki has not provided any of the requested source code.

There are other Trident+ switches (like the Arista DCS-7050S) that can be purchased for around $200 USD, and do not require any additional effort to use. Unless you have decommissioned MS420 switches, I would recommend against buying one as there are better options available.

Model Meraki Board Part number
MS420-24 FATBOY 600-29020
MS420-48 FATBOY 600-29010

Breaking secure boot on the Meraki Z3 and Meraki Go GX20

Leave a reply

Meraki used to be friendly to open-source. Older devices and Meraki firmwares offered a root shell over UART, and it was relatively easy to flash custom firmware to the device. Later, they began locking down devices so that people could no longer repurpose the hardware. Their latest “innovations” include the use of secure boot, ostensibly to improve device security but with the side effect of making claimed and EoL devices e-waste.

“Z3 is a remote work gateway with integrated Wi-Fi, secure access to corporate and multi-cloud resources, and features that delight SD-branch cloud platform users.”

Today we are looking at the Meraki Z3 and Meraki Go GX20. Both the Z3 and GX20 ship with secure boot enabled, meaning you cannot boot an alternative firmware like OpenWrt on the devices, and they are useless if the previous owner has not “unclaimed” them in Meraki’s dashboard.

Meraki chose to use the same u-boot release for their Qualcomm IPQ40xx based devices (MR33, MR30H, Z3, Go GX20). The MR33 ships without secure boot. However, since secure boot is enabled on the Z3 and Meraki Go GX20, u-boot on those devices must be signed with a certificate matching the one burned into the QFPROM fuses, or the device will not boot. In the case of the Z3, u-boot is signed with the fuzzy-cricket attestation ca1 certificate:

California1
San Francisco
Cisco Meraki1%0#
fuzzy-cricket attestation ca1
Product Security0
170728184745Z
170827184745Z0

The boot flow on an ipq40xx device with secure boot enabled can be summarized in the following diagram (adapted from the LineageOS docs):

Since Meraki use the same u-boot for devices with and without secure boot, u-boot must determine at some point whether it should enforce signature verification on the payload (a Flattened Image Tree Image). That check happens here:

static int do_meraki_qca_boot(cmd_tbl_t * cmdtp, int flag, int argc, char * const argv[])
{
   /* unsupported boards */
   switch(get_meraki_product_id()) {
      case MERAKI_BOARD_STINKBUG:
      case MERAKI_BOARD_LADYBUG:
      case MERAKI_BOARD_NOISY_CRICKET:
      case MERAKI_BOARD_YOWIE:
      case MERAKI_BOARD_BIGFOOT:
      case MERAKI_BOARD_SASQUATCH:
      case MERAKI_BOARD_WOOKIE:
         return 0;
      default:
         break;
   }

   /* Check 0: check/force secure boot on */
   force_secboot();

   /* Check 1: boot diagnostic */
   run_command("run check_boot_diag_img", 0);

   /* Check 2: boot part.new if upgrading */
   do_upgrade_boot();

   /* Check 3: if node specific unlock exists, chainload u-boot */
   if (devel_crt_valid()) {
      do_chainload_uboot();
   }

   setenv("part","part.safe");
   run_command("run boot_signedpart",0);

   setenv("part","part.old");
   run_command("run boot_signedpart",0);

   run_command("reset",0);

   return 0;
}

Note that if the board is STINKBUG (MR33), LADYBUG (MR74), NOISY_CRICKET (MR30H), YOWIE (MR42), BIGFOOT (MR52), SASQUATCH (MR53), or WOOKIE (MR84) then verification of the FIT image is skipped, as these devices don’t have secure boot enabled by default.

The 2017.07 u-boot update for the MR33 is a special case, as that doesn’t actually enable secure boot, it just maliciously bricks the device if the user attempts to interrupt boot.

How this practice is permitted under the EU Unfair Commercial Practices Directive (Directive 2005/29/EC of 2005) is quite frankly a mystery to me, but perhaps someone with more legal expertise can weigh in.

u-boot determines the device it is running on by reading the “Board major number” from an I2C EEPROM (24c64; silkscreen U32) on the PCB. Modifying the contents of the EEPROM from a device model with secure boot to a device without secure boot is enough to disable signature verification of payloads in u-boot. You can find the “board major number” byte at offset 0x49 and a backup copy at 0x1049 in the EEPROM.

Original Z3 boot output:

Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.1-00096
S - IMAGE_VARIANT_STRING=DAACANAZA
S - OEM_IMAGE_VERSION_STRING=CRM
S - Boot Config, 0x00000025
S - Core 0 Frequency, 0 MHz
B -       261 - PBL, Start
B -      1340 - bootable_media_detect_entry, Start
B -      2615 - bootable_media_detect_success, Start
B -      2629 - elf_loader_entry, Start
B -      7204 - auth_hash_seg_entry, Start
B -   1382243 - auth_hash_seg_exit, Start
B -   1447600 - elf_segs_hash_verify_entry, Start
B -   1569288 - PBL, End
B -   1569312 - SBL1, Start
B -   1657984 - pm_device_init, Start
D -         6 - pm_device_init, Delta
B -   1659502 - boot_flash_init, Start
D -     87535 - boot_flash_init, Delta
B -   1751079 - boot_config_data_table_init, Start
D -     14006 - boot_config_data_table_init, Delta - (419 Bytes)
B -   1767784 - clock_init, Start
D -      7575 - clock_init, Delta
B -   1778758 - CDT version:2,Platform ID:8,Major ID:1,Minor ID:0,Subtype:1
B -   1782246 - sbl1_ddr_set_params, Start
B -   1787231 - cpr_init, Start
D -         2 - cpr_init, Delta
B -   1791720 - Pre_DDR_clock_init, Start
D -         5 - Pre_DDR_clock_init, Delta
D -     13140 - sbl1_ddr_set_params, Delta
B -   1804998 - pm_driver_init, Start
D -         2 - pm_driver_init, Delta
B -   1875945 - sbl1_wait_for_ddr_training, Start
D -        27 - sbl1_wait_for_ddr_training, Delta
B -   1893463 - Image Load, Start
D -   1311048 - QSEE Image Loaded, Delta - (268504 Bytes)
B -   3205010 - Image Load, Start
D -      2116 - SEC Image Loaded, Delta - (2048 Bytes)
B -   3215195 - Image Load, Start
D -   1307290 - APPSBL Image Loaded, Delta - (292616 Bytes)
B -   4522910 - QSEE Execution, Start
D -        56 - QSEE Execution, Delta
B -   4529089 - SBL1, End
D -   2961858 - SBL1, Delta
S - Flash Throughput, 1970 KB/s  (563587 Bytes,  285975 us)
S - DDR Frequency, 672 MHz


U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Fuzzy_Cricket
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
ubi0: attaching mtd1
ubi0: scanning is finished
ubi0: attached mtd1 (name "mtd=0", size 112 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 896, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 5, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 124/51, WL threshold: 4096, image sequence number: 1218587189
ubi0: available PEBs: 337, total reserved PEBs: 559, PEBs reserved for bad PEB handling: 20


Secure boot enabled.

Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (16584704)
Valid image
## Loading kernel from FIT Image at 84000028 ...
   Using 'config@1' configuration
   Trying 'kernel@1' kernel subimage
     Description:  wired-arm-qca Kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x84000134
     Data Size:    1962384 Bytes = 1.9 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x80208000
     Entry Point:  0x80208000
     Hash algo:    sha1
     Hash value:   1a716f7999511396baa166ef3986165f40c4c1c7
   Verifying Hash Integrity ... sha1+ OK
## Loading ramdisk from FIT Image at 84000028 ...
   Using 'config@1' configuration
   Trying 'ramdisk@1' ramdisk subimage
     Description:  wired-arm-qca Ramdisk
     Type:         RAMDisk Image
     Compression:  uncompressed
     Data Start:   0x841df3b8
     Data Size:    14496712 Bytes = 13.8 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x82200000
     Entry Point:  0x82200000
     Hash algo:    sha1
     Hash value:   6c03ebf4feff11ed19dadcd2b6afe329d74e6671
   Verifying Hash Integrity ... sha1+ OK
   Loading ramdisk from 0x841df3b8 to 0x82200000
## Loading fdt from FIT Image at 84000028 ...
   Using 'config@1' configuration
   Trying 'fdt@1' fdt subimage
     Description:  Fuzzy Cricket Device Tree
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x84fb2874
     Data Size:    38216 Bytes = 37.3 KiB
     Architecture: ARM
     Hash algo:    sha1
     Hash value:   25733212c52b5e5803c8fe02a64fd229f30a2ac4
   Verifying Hash Integrity ... sha1+ OK
   Loading fdt from 0x84fb2874 to 0x89000000
   Booting using the fdt blob at 0x89000000
   Loading Kernel Image ... OK
   Using Device Tree in place at 89000000, end 8900c547
Using machid 0x8010001 from environment

Starting kernel ...

After modifying the Z3 EEPROM to have the board major number of the MR33:

Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.1-00096
S - IMAGE_VARIANT_STRING=DAACANAZA
S - OEM_IMAGE_VERSION_STRING=CRM
S - Boot Config, 0x00000025
S - Core 0 Frequency, 0 MHz
B -       261 - PBL, Start
B -      1340 - bootable_media_detect_entry, Start
B -      2615 - bootable_media_detect_success, Start
B -      2629 - elf_loader_entry, Start
B -      7204 - auth_hash_seg_entry, Start
B -   1382292 - auth_hash_seg_exit, Start
B -   1447925 - elf_segs_hash_verify_entry, Start
B -   1569633 - PBL, End
B -   1569657 - SBL1, Start
B -   1658338 - pm_device_init, Start
D -         6 - pm_device_init, Delta
B -   1659859 - boot_flash_init, Start
D -     87566 - boot_flash_init, Delta
B -   1751466 - boot_config_data_table_init, Start
D -     14002 - boot_config_data_table_init, Delta - (419 Bytes)
B -   1768163 - clock_init, Start
D -      7570 - clock_init, Delta
B -   1779133 - CDT version:2,Platform ID:8,Major ID:1,Minor ID:0,Subtype:1
B -   1782622 - sbl1_ddr_set_params, Start
B -   1787606 - cpr_init, Start
D -         2 - cpr_init, Delta
B -   1792095 - Pre_DDR_clock_init, Start
D -         5 - Pre_DDR_clock_init, Delta
D -     13141 - sbl1_ddr_set_params, Delta
B -   1805373 - pm_driver_init, Start
D -         2 - pm_driver_init, Delta
B -   1876249 - sbl1_wait_for_ddr_training, Start
D -        27 - sbl1_wait_for_ddr_training, Delta
B -   1893981 - Image Load, Start
D -   1311108 - QSEE Image Loaded, Delta - (268504 Bytes)
B -   3205588 - Image Load, Start
D -      2122 - SEC Image Loaded, Delta - (2048 Bytes)
B -   3215674 - Image Load, Start
D -   1307310 - APPSBL Image Loaded, Delta - (292616 Bytes)
B -   4523411 - QSEE Execution, Start
D -        56 - QSEE Execution, Delta
B -   4529587 - SBL1, End
D -   2962011 - SBL1, Delta
S - Flash Throughput, 1969 KB/s  (563587 Bytes,  286173 us)
S - DDR Frequency, 672 MHz


U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
ubi0: attaching mtd1
ubi0: scanning is finished
ubi0: attached mtd1 (name "mtd=0", size 112 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 896, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 5, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 124/52, WL threshold: 4096, image sequence number: 1218587189
ubi0: available PEBs: 337, total reserved PEBs: 559, PEBs reserved for bad PEB handling: 20
Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (16584704)
Wrong Image Format for bootm command
ERROR: can't get kernel image!
Read 0 bytes from volume part.old to 84000000
No size specified -> Using max size (16547840)
Wrong Image Format for bootm command
ERROR: can't get kernel image!
resetting ...

Note the change in output, the device was previously meraki_Fuzzy_Cricket but is now identified as meraki_Stinkbug, so the EEPROM modification worked. Unfortunately, the format of the FIT images differs between secure boot and non-secure boot, so the device no longer boots the stock firmware.

The MR33 board major number was chosen as u-boot selects config@1 from the FIT image for the MR33, which is the same config index used by default for the Z3. It would be possible to select any of the device models mentioned above that don’t implement secure boot, but doing so would mean u-boot attempts to boot from a different config@ entry.

Since the Z3 and Meraki Go GX20 have secure boot enabled, we cannot replace u-boot or the device will fail to boot.

0x000000000000-0x000000100000 : "sbl1"
0x000000100000-0x000000200000 : "mibib"
0x000000200000-0x000000300000 : "bootconfig"
0x000000300000-0x000000400000 : "qsee"
0x000000400000-0x000000500000 : "qsee_alt"
0x000000500000-0x000000580000 : "cdt"
0x000000580000-0x000000600000 : "cdt_alt"
0x000000600000-0x000000680000 : "ddrparams"
0x000000700000-0x000000900000 : "u-boot"
0x000000900000-0x000000b00000 : "u-boot-backup"
0x000000b00000-0x000000b80000 : "ART"
0x000000c00000-0x000007c00000 : "ubi"

However, by modifying the device major number, the (signed) u-boot will now allow booting unsigned payloads. The easiest way to proceed is to then build our own modified u-boot as a FIT image, and put it into the ubivol part.safe. After we do this, the boot process now looks like:

Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.1-00096
S - IMAGE_VARIANT_STRING=DAACANAZA
S - OEM_IMAGE_VERSION_STRING=CRM
S - Boot Config, 0x00000025
S - Core 0 Frequency, 0 MHz
B -       261 - PBL, Start
B -      1340 - bootable_media_detect_entry, Start
B -      2615 - bootable_media_detect_success, Start
B -      2629 - elf_loader_entry, Start
B -      7270 - auth_hash_seg_entry, Start
B -   1382352 - auth_hash_seg_exit, Start
B -   1447639 - elf_segs_hash_verify_entry, Start
B -   1569353 - PBL, End
B -   1569377 - SBL1, Start
B -   1658051 - pm_device_init, Start
D -         6 - pm_device_init, Delta
B -   1659570 - boot_flash_init, Start
D -     87594 - boot_flash_init, Delta
B -   1751205 - boot_config_data_table_init, Start
D -     14010 - boot_config_data_table_init, Delta - (419 Bytes)
B -   1767914 - clock_init, Start
D -      7572 - clock_init, Delta
B -   1778887 - CDT version:2,Platform ID:8,Major ID:1,Minor ID:0,Subtype:1
B -   1782376 - sbl1_ddr_set_params, Start
B -   1787361 - cpr_init, Start
D -         2 - cpr_init, Delta
B -   1791851 - Pre_DDR_clock_init, Start
D -         5 - Pre_DDR_clock_init, Delta
D -     13143 - sbl1_ddr_set_params, Delta
B -   1805130 - pm_driver_init, Start
D -         2 - pm_driver_init, Delta
B -   1876340 - sbl1_wait_for_ddr_training, Start
D -        27 - sbl1_wait_for_ddr_training, Delta
B -   1893870 - Image Load, Start
D -   1312117 - QSEE Image Loaded, Delta - (268504 Bytes)
B -   3206486 - Image Load, Start
D -      2118 - SEC Image Loaded, Delta - (2048 Bytes)
B -   3216578 - Image Load, Start
D -   1308369 - APPSBL Image Loaded, Delta - (292616 Bytes)
B -   4525372 - QSEE Execution, Start
D -        56 - QSEE Execution, Delta
B -   4531549 - SBL1, End
D -   2964253 - SBL1, Delta
S - Flash Throughput, 1969 KB/s  (563587 Bytes,  286154 us)
S - DDR Frequency, 672 MHz


U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
ubi0: attaching mtd1
ubi0: scanning is finished
ubi0: attached mtd1 (name "mtd=0", size 112 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 896, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 5, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 214/99, WL threshold: 4096, image sequence number: 2086049366
ubi0: available PEBs: 0, total reserved PEBs: 896, PEBs reserved for bad PEB handling: 20
Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (507904)
## Loading kernel from FIT Image at 84000000 ...
   Using 'config@1' configuration
   Trying 'kernel-1' kernel subimage
     Description:  Kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x840000d0
     Data Size:    381216 Bytes = 372.3 KiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x87300000
     Entry Point:  0x87300000
     Hash algo:    sha1
     Hash value:   89c319b76738e71147631f87311fc8f31e8ac8aa
   Verifying Hash Integrity ... sha1+ OK
## Loading fdt from FIT Image at 84000000 ...
   Using 'config@1' configuration
   Trying 'fdt-1' fdt subimage
     Description:  Insect DTB
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x8405d2d4
     Data Size:    235 Bytes = 235 Bytes
     Architecture: ARM
     Hash algo:    sha1
     Hash value:   86c47255d86f2bd6301e7772ca65b3548493875b
   Verifying Hash Integrity ... sha1+ OK
   Booting using the fdt blob at 0x8405d2d4
   Loading Kernel Image ... OK
   Using Device Tree in place at 8405d2d4, end 840603be
Using machid 0x8010001 from environment

Starting kernel ...



U-Boot 2017.07-DEVEL (Apr 01 2024 - 14:50:01 +0000)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
Net:   MAC0 addr:e0:cb:bc:11:22:33
PHY ID1: 0x4d
PHY ID2: 0xd0b1
ipq40xx_ess_sw_init done
eth0
Autoboot in 5 seconds
FUZZY CRICKET #

We could proceed by putting OpenWrt directly in part.safe however if we ever had a bad flash there would be no way to recover except by manually reflashing NAND. Chain loading a u-boot build does delay booting by a few seconds, but offers a way to recover from a bad flash with just UART and tftpboot.

With the modified u-boot on flash, we can now tftpboot the OpenWrt initramfs and install it to flash with sysupgrade.

“Meraki Go offers simple networks for serious business.”

The Meraki Go GX20 uses the same PCB as the Z3, but without the WiFi radios populated (the lack of WiFi means you won’t find the GX20 in the FCC database). So, does the above technique work for the Meraki Go GX20?

U-Boot 2012.07-g03cdfe19e00f [local,local] (Aug 29 2017 - 11:59:45)

DRAM:  498 MiB
machid : 0x8010001
ERROR: Unknown board
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
Hit any key to stop autoboot:  0
Creating 1 MTD partitions on "nand0":
0x000000c00000-0x000007f00000 : "mtd=0"
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    126976 bytes
UBI: smallest flash I/O unit:    2048
UBI: VID header offset:          2048 (aligned 2048)
UBI: data offset:                4096
UBI: attached mtd1 to ubi0
UBI: MTD device name:            "mtd=0"
UBI: MTD device size:            115 MiB
UBI: number of good PEBs:        920
UBI: number of bad PEBs:         0
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    4096
UBI: number of internal volumes: 1
UBI: number of user volumes:     7
UBI: available PEBs:             364
UBI: total number of reserved PEBs: 556
UBI: number of PEBs reserved for bad PEB handling: 9
UBI: max/mean erase counter: 2/0
Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (3284992)
## Booting kernel from FIT Image at 84000000 ...
   Using 'config@1' configuration
   Verifying Hash Integrity ... sha384,secp384r1:wired-arm-qca-RT-SECP384R1_1-rel+ OK
   Trying 'kernel@1' kernel subimage

No, unfortunately for u-boot 2012.07 which shipped on the Meraki Go GX20 I purchased, changing the device major results in ERROR: Unknown board and we can see from the sha384,secp384r1:wired-arm-qca-RT-SECP384R1_1-rel+ OK output that signature checking is still enabled.

However, the Meraki Go GX20 is a very similar device to the Z3. Is it possible that there is another way?

Examining the u-boot region of the Meraki Go GX20 with strings reveals the following:

California1
San Francisco
Cisco Meraki1%0#
fuzzy-cricket attestation ca1
Product Security0
170728184745Z
170827184745Z0

So the u-boot binary on the Meraki Go GX20 is signed with the same signing cert as u-boot on the Z3. What if we replace u-boot on the Meraki Go GX20 with the signed u-boot from the Z3 dump?

U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

Bingo. And since the Z3 2017.07 u-boot release contains the secure boot downgrade bug, we can exploit it on the Meraki Go GX20 as well.

I am curious to see how u-boot 2012.07 on the Meraki Go GX20 implements signature validation, since unlike u-boot 2017.07 changing the device major does not disable signature verification of the payload.

Unfortunately, this is not possible. The GPL source code for the Meraki Go GX20 was requested in December 2022 and Meraki has yet to provide the u-boot source code for the device.

The EU and other jurisdictions claim to take the right to repair and e-waste seriously, but their actions thus far have ignored the elephant in the room. Many of the devices being sold in the past 5 years come with secure boot enabled, and thus are locked to running the OEM’s software.

In the case of the Meraki prodcuts, you have purchased the device and Meraki are selling a license only for the cloud management and hardware replacement service. But, if you stop paying Meraki, buy a “claimed” second-hand device, or Meraki discontinue support for the device you now have a worthless brick.

“Cisco Meraki may find it necessary to discontinue products for a number of reasons, including product line enhancements, market demand, technology innovation, or if the product simply matures over time and needs to be replaced by something functionally richer.”

To be clear, you are not leasing the hardware, and this is not “Hardware as a Service” you have bought the device in question and own it.

There are hundreds of SMD resistors on the device’s PCB. Any one of them could be tied to a GPIO which is polled by the SoC BootROM to enable or disable secure boot. Incorporating this into the hardware design would cost nothing, and would allow consumers the choice to re-use devices that were resold without proper decommissioning, from companies that were liquidated, from devices that were recycled as e-waste, or from devices where the manufacturer has decided to “maximize shareholder value” and end support. Physically opening the device and adding or removing a component to disable secure boot does not compromise user security in any way.

Routers like the Z3 and Meraki Go GX20 may appear to the untrained eye to be highly specialized devices, but in reality they are very similar to a Raspberry Pi with additional network interfaces, e.g. they are general purpose embedded computers.

It is past time that regulators start considering the user-harmful practices of companies in enforcing secure boot on these devices with no way for consumers to exercise their rights. This anti-consumer behaviour increases costs and creates more e-waste, solely for the benefit of the bottom line of companies that are (often) not even based in the same economic area. We should not, and cannot, accept such a status quo. There is no reason for these devices to become e-waste!

Impact: Device owners with physical access to their device and the appropriate hardware flashing tools can install a custom firmware.

Could this be accomplished remotely? No, not unless it is chained with another exploit providing Remote Code Execution (RCE) on the device. Anyone with an RCE on Meraki devices would not waste it on flashing a FOSS firmware, they would use it to build a botnet.

Can Meraki patch this? Yes, see below it is trivial for them to add a check in u-boot 2017.07 whether secure boot is enabled and enforce signature verification. Patching this is left as an exercise to the reader.

Will they patch it? Meraki announced the EoL of the Z3 in March 2024. They have previously been rumoured to update u-boot to brick devices when users attempt to flash a custom firmware. It remains to be seen if they will continue this anti-consumer practice on the Z3 and Meraki Go GX20.

Responsible disclosure: Yes, this has been responsibly disclosed to you, the device owner.

2024-06: The above method has also been confirmed to bypass FIT signature verification on the Meraki Z3C.

2024-07: Meraki inadvertently addressed this vulnerability in U-Boot builds dated U-Boot 2017.07-RELEASE-gf49d105aeb-dirty (Jul 13 2020 - 11:22:51 -0700) and later when they removed support for devices without secure boot (MR33, MR30H, MR74).