Tag Archives: openwrt

Breaking secure boot on the Meraki Z3 and Meraki Go GX20

Meraki used to be friendly to open-source. Older devices and Meraki firmwares offered a root shell over UART, and it was relatively easy to flash custom firmware to the device. Later, they began locking down devices so that people could no longer repurpose the hardware. Their latest “innovations” include the use of secure boot, ostensibly to improve device security but with the side effect of making claimed and EoL devices e-waste.


“Z3 is a remote work gateway with integrated Wi-Fi, secure access to corporate and multi-cloud resources, and features that delight SD-branch cloud platform users.”

Today we are looking at the Meraki Z3 and Meraki Go GX20. Both the Z3 and GX20 ship with secure boot enabled, meaning you cannot boot an alternative firmware like OpenWrt on the devices, and they are useless if the previous owner has not “unclaimed” them in Meraki’s dashboard.

Meraki chose to use the same u-boot release for their Qualcomm IPQ40xx based devices (MR33, MR30H, Z3, Go GX20). The MR33 ships without secure boot. However, since secure boot is enabled on the Z3 and Meraki Go GX20, u-boot on those devices must be signed with a certificate matching the one burned into the QFPROM fuses, or the device will not boot. In the case of the Z3, u-boot is signed with the fuzzy-cricket attestation ca1 certificate:

California1
San Francisco
Cisco Meraki1%0#
fuzzy-cricket attestation ca1
Product Security0
170728184745Z
170827184745Z0

The boot flow on an ipq40xx device with secure boot enabled can be summarized in the following diagram (adapted from the LineageOS docs):

Since Meraki use the same u-boot for devices with and without secure boot, u-boot must determine at some point whether it should enforce signature verification on the payload (a Flattened Image Tree Image). That check happens here:

static int do_meraki_qca_boot(cmd_tbl_t * cmdtp, int flag, int argc, char * const argv[])
{
   /* unsupported boards */
   switch(get_meraki_product_id()) {
      case MERAKI_BOARD_STINKBUG:
      case MERAKI_BOARD_LADYBUG:
      case MERAKI_BOARD_NOISY_CRICKET:
      case MERAKI_BOARD_YOWIE:
      case MERAKI_BOARD_BIGFOOT:
      case MERAKI_BOARD_SASQUATCH:
      case MERAKI_BOARD_WOOKIE:
         return 0;
      default:
         break;
   }

   /* Check 0: check/force secure boot on */
   force_secboot();

   /* Check 1: boot diagnostic */
   run_command("run check_boot_diag_img", 0);

   /* Check 2: boot part.new if upgrading */
   do_upgrade_boot();

   /* Check 3: if node specific unlock exists, chainload u-boot */
   if (devel_crt_valid()) {
      do_chainload_uboot();
   }

   setenv("part","part.safe");
   run_command("run boot_signedpart",0);

   setenv("part","part.old");
   run_command("run boot_signedpart",0);

   run_command("reset",0);

   return 0;
}

Note that if the board is STINKBUG (MR33), LADYBUG (MR74), NOISY_CRICKET (MR30H), YOWIE (MR42), BIGFOOT (MR52), SASQUATCH (MR53), or WOOKIE (MR84) then verification of the FIT image is skipped, as these devices don’t have secure boot enabled by default.


The 2017.07 u-boot update for the MR33 is a special case, as that doesn’t actually enable secure boot, it just maliciously bricks the device if the user attempts to interrupt boot.

How this practice is permitted under the EU Unfair Commercial Practices Directive (Directive 2005/29/EC of 2005) is quite frankly a mystery to me, but perhaps someone with more legal expertise can weigh in.


u-boot determines which devices its running on by reading the “Board major number” from an I2C EEPROM (24c64; silkscreen U32) on the PCB. Modifying the contents of the EEPROM from a device model with secure boot to a device without secure boot is enough to disable signature verification of payloads in u-boot. You can find the “board major number” byte at offset 0x4a and a backup copy at 0x104a in the EEPROM.

Original Z3 boot output:

Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.1-00096
S - IMAGE_VARIANT_STRING=DAACANAZA
S - OEM_IMAGE_VERSION_STRING=CRM
S - Boot Config, 0x00000025
S - Core 0 Frequency, 0 MHz
B -       261 - PBL, Start
B -      1340 - bootable_media_detect_entry, Start
B -      2615 - bootable_media_detect_success, Start
B -      2629 - elf_loader_entry, Start
B -      7204 - auth_hash_seg_entry, Start
B -   1382243 - auth_hash_seg_exit, Start
B -   1447600 - elf_segs_hash_verify_entry, Start
B -   1569288 - PBL, End
B -   1569312 - SBL1, Start
B -   1657984 - pm_device_init, Start
D -         6 - pm_device_init, Delta
B -   1659502 - boot_flash_init, Start
D -     87535 - boot_flash_init, Delta
B -   1751079 - boot_config_data_table_init, Start
D -     14006 - boot_config_data_table_init, Delta - (419 Bytes)
B -   1767784 - clock_init, Start
D -      7575 - clock_init, Delta
B -   1778758 - CDT version:2,Platform ID:8,Major ID:1,Minor ID:0,Subtype:1
B -   1782246 - sbl1_ddr_set_params, Start
B -   1787231 - cpr_init, Start
D -         2 - cpr_init, Delta
B -   1791720 - Pre_DDR_clock_init, Start
D -         5 - Pre_DDR_clock_init, Delta
D -     13140 - sbl1_ddr_set_params, Delta
B -   1804998 - pm_driver_init, Start
D -         2 - pm_driver_init, Delta
B -   1875945 - sbl1_wait_for_ddr_training, Start
D -        27 - sbl1_wait_for_ddr_training, Delta
B -   1893463 - Image Load, Start
D -   1311048 - QSEE Image Loaded, Delta - (268504 Bytes)
B -   3205010 - Image Load, Start
D -      2116 - SEC Image Loaded, Delta - (2048 Bytes)
B -   3215195 - Image Load, Start
D -   1307290 - APPSBL Image Loaded, Delta - (292616 Bytes)
B -   4522910 - QSEE Execution, Start
D -        56 - QSEE Execution, Delta
B -   4529089 - SBL1, End
D -   2961858 - SBL1, Delta
S - Flash Throughput, 1970 KB/s  (563587 Bytes,  285975 us)
S - DDR Frequency, 672 MHz


U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Fuzzy_Cricket
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
ubi0: attaching mtd1
ubi0: scanning is finished
ubi0: attached mtd1 (name "mtd=0", size 112 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 896, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 5, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 124/51, WL threshold: 4096, image sequence number: 1218587189
ubi0: available PEBs: 337, total reserved PEBs: 559, PEBs reserved for bad PEB handling: 20


Secure boot enabled.

Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (16584704)
Valid image
## Loading kernel from FIT Image at 84000028 ...
   Using 'config@1' configuration
   Trying 'kernel@1' kernel subimage
     Description:  wired-arm-qca Kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x84000134
     Data Size:    1962384 Bytes = 1.9 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x80208000
     Entry Point:  0x80208000
     Hash algo:    sha1
     Hash value:   1a716f7999511396baa166ef3986165f40c4c1c7
   Verifying Hash Integrity ... sha1+ OK
## Loading ramdisk from FIT Image at 84000028 ...
   Using 'config@1' configuration
   Trying 'ramdisk@1' ramdisk subimage
     Description:  wired-arm-qca Ramdisk
     Type:         RAMDisk Image
     Compression:  uncompressed
     Data Start:   0x841df3b8
     Data Size:    14496712 Bytes = 13.8 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x82200000
     Entry Point:  0x82200000
     Hash algo:    sha1
     Hash value:   6c03ebf4feff11ed19dadcd2b6afe329d74e6671
   Verifying Hash Integrity ... sha1+ OK
   Loading ramdisk from 0x841df3b8 to 0x82200000
## Loading fdt from FIT Image at 84000028 ...
   Using 'config@1' configuration
   Trying 'fdt@1' fdt subimage
     Description:  Fuzzy Cricket Device Tree
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x84fb2874
     Data Size:    38216 Bytes = 37.3 KiB
     Architecture: ARM
     Hash algo:    sha1
     Hash value:   25733212c52b5e5803c8fe02a64fd229f30a2ac4
   Verifying Hash Integrity ... sha1+ OK
   Loading fdt from 0x84fb2874 to 0x89000000
   Booting using the fdt blob at 0x89000000
   Loading Kernel Image ... OK
   Using Device Tree in place at 89000000, end 8900c547
Using machid 0x8010001 from environment

Starting kernel ...

After modifying the Z3 EEPROM to have the board major number of the MR33:

Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.1-00096
S - IMAGE_VARIANT_STRING=DAACANAZA
S - OEM_IMAGE_VERSION_STRING=CRM
S - Boot Config, 0x00000025
S - Core 0 Frequency, 0 MHz
B -       261 - PBL, Start
B -      1340 - bootable_media_detect_entry, Start
B -      2615 - bootable_media_detect_success, Start
B -      2629 - elf_loader_entry, Start
B -      7204 - auth_hash_seg_entry, Start
B -   1382292 - auth_hash_seg_exit, Start
B -   1447925 - elf_segs_hash_verify_entry, Start
B -   1569633 - PBL, End
B -   1569657 - SBL1, Start
B -   1658338 - pm_device_init, Start
D -         6 - pm_device_init, Delta
B -   1659859 - boot_flash_init, Start
D -     87566 - boot_flash_init, Delta
B -   1751466 - boot_config_data_table_init, Start
D -     14002 - boot_config_data_table_init, Delta - (419 Bytes)
B -   1768163 - clock_init, Start
D -      7570 - clock_init, Delta
B -   1779133 - CDT version:2,Platform ID:8,Major ID:1,Minor ID:0,Subtype:1
B -   1782622 - sbl1_ddr_set_params, Start
B -   1787606 - cpr_init, Start
D -         2 - cpr_init, Delta
B -   1792095 - Pre_DDR_clock_init, Start
D -         5 - Pre_DDR_clock_init, Delta
D -     13141 - sbl1_ddr_set_params, Delta
B -   1805373 - pm_driver_init, Start
D -         2 - pm_driver_init, Delta
B -   1876249 - sbl1_wait_for_ddr_training, Start
D -        27 - sbl1_wait_for_ddr_training, Delta
B -   1893981 - Image Load, Start
D -   1311108 - QSEE Image Loaded, Delta - (268504 Bytes)
B -   3205588 - Image Load, Start
D -      2122 - SEC Image Loaded, Delta - (2048 Bytes)
B -   3215674 - Image Load, Start
D -   1307310 - APPSBL Image Loaded, Delta - (292616 Bytes)
B -   4523411 - QSEE Execution, Start
D -        56 - QSEE Execution, Delta
B -   4529587 - SBL1, End
D -   2962011 - SBL1, Delta
S - Flash Throughput, 1969 KB/s  (563587 Bytes,  286173 us)
S - DDR Frequency, 672 MHz


U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
ubi0: attaching mtd1
ubi0: scanning is finished
ubi0: attached mtd1 (name "mtd=0", size 112 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 896, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 5, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 124/52, WL threshold: 4096, image sequence number: 1218587189
ubi0: available PEBs: 337, total reserved PEBs: 559, PEBs reserved for bad PEB handling: 20
Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (16584704)
Wrong Image Format for bootm command
ERROR: can't get kernel image!
Read 0 bytes from volume part.old to 84000000
No size specified -> Using max size (16547840)
Wrong Image Format for bootm command
ERROR: can't get kernel image!
resetting ...

Note the change in output, the device was previously meraki_Fuzzy_Cricket but is now identified as meraki_Stinkbug, so the EEPROM modification worked. Unfortunately, the format of the FIT images differs between secure boot and non-secure boot, so the device no longer boots the stock firmware.

The MR33 board major number was chosen as u-boot selects config@1 from the FIT image for the MR33, which is the same config index used by default for the Z3. It would be possible to select any of the device models mentioned above that don’t implement secure boot, but doing so would mean u-boot attempts to boot from a different config@ entry.

Since the Z3 and Meraki Go GX20 have secure boot enabled, we cannot replace u-boot or the device will fail to boot.

0x000000000000-0x000000100000 : "sbl1"
0x000000100000-0x000000200000 : "mibib"
0x000000200000-0x000000300000 : "bootconfig"
0x000000300000-0x000000400000 : "qsee"
0x000000400000-0x000000500000 : "qsee_alt"
0x000000500000-0x000000580000 : "cdt"
0x000000580000-0x000000600000 : "cdt_alt"
0x000000600000-0x000000680000 : "ddrparams"
0x000000700000-0x000000900000 : "u-boot"
0x000000900000-0x000000b00000 : "u-boot-backup"
0x000000b00000-0x000000b80000 : "ART"
0x000000c00000-0x000007c00000 : "ubi"

However, by modifying the device major number, the (signed) u-boot will now allow booting unsigned payloads. The easiest way to proceed is to then build our own modified u-boot as a FIT image, and put it into the ubivol part.safe. After we do this, the boot process now looks like:

Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.1-00096
S - IMAGE_VARIANT_STRING=DAACANAZA
S - OEM_IMAGE_VERSION_STRING=CRM
S - Boot Config, 0x00000025
S - Core 0 Frequency, 0 MHz
B -       261 - PBL, Start
B -      1340 - bootable_media_detect_entry, Start
B -      2615 - bootable_media_detect_success, Start
B -      2629 - elf_loader_entry, Start
B -      7270 - auth_hash_seg_entry, Start
B -   1382352 - auth_hash_seg_exit, Start
B -   1447639 - elf_segs_hash_verify_entry, Start
B -   1569353 - PBL, End
B -   1569377 - SBL1, Start
B -   1658051 - pm_device_init, Start
D -         6 - pm_device_init, Delta
B -   1659570 - boot_flash_init, Start
D -     87594 - boot_flash_init, Delta
B -   1751205 - boot_config_data_table_init, Start
D -     14010 - boot_config_data_table_init, Delta - (419 Bytes)
B -   1767914 - clock_init, Start
D -      7572 - clock_init, Delta
B -   1778887 - CDT version:2,Platform ID:8,Major ID:1,Minor ID:0,Subtype:1
B -   1782376 - sbl1_ddr_set_params, Start
B -   1787361 - cpr_init, Start
D -         2 - cpr_init, Delta
B -   1791851 - Pre_DDR_clock_init, Start
D -         5 - Pre_DDR_clock_init, Delta
D -     13143 - sbl1_ddr_set_params, Delta
B -   1805130 - pm_driver_init, Start
D -         2 - pm_driver_init, Delta
B -   1876340 - sbl1_wait_for_ddr_training, Start
D -        27 - sbl1_wait_for_ddr_training, Delta
B -   1893870 - Image Load, Start
D -   1312117 - QSEE Image Loaded, Delta - (268504 Bytes)
B -   3206486 - Image Load, Start
D -      2118 - SEC Image Loaded, Delta - (2048 Bytes)
B -   3216578 - Image Load, Start
D -   1308369 - APPSBL Image Loaded, Delta - (292616 Bytes)
B -   4525372 - QSEE Execution, Start
D -        56 - QSEE Execution, Delta
B -   4531549 - SBL1, End
D -   2964253 - SBL1, Delta
S - Flash Throughput, 1969 KB/s  (563587 Bytes,  286154 us)
S - DDR Frequency, 672 MHz


U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
ubi0: attaching mtd1
ubi0: scanning is finished
ubi0: attached mtd1 (name "mtd=0", size 112 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 896, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 5, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 214/99, WL threshold: 4096, image sequence number: 2086049366
ubi0: available PEBs: 0, total reserved PEBs: 896, PEBs reserved for bad PEB handling: 20
Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (507904)
## Loading kernel from FIT Image at 84000000 ...
   Using 'config@1' configuration
   Trying 'kernel-1' kernel subimage
     Description:  Kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x840000d0
     Data Size:    381216 Bytes = 372.3 KiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x87300000
     Entry Point:  0x87300000
     Hash algo:    sha1
     Hash value:   89c319b76738e71147631f87311fc8f31e8ac8aa
   Verifying Hash Integrity ... sha1+ OK
## Loading fdt from FIT Image at 84000000 ...
   Using 'config@1' configuration
   Trying 'fdt-1' fdt subimage
     Description:  Insect DTB
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x8405d2d4
     Data Size:    235 Bytes = 235 Bytes
     Architecture: ARM
     Hash algo:    sha1
     Hash value:   86c47255d86f2bd6301e7772ca65b3548493875b
   Verifying Hash Integrity ... sha1+ OK
   Booting using the fdt blob at 0x8405d2d4
   Loading Kernel Image ... OK
   Using Device Tree in place at 8405d2d4, end 840603be
Using machid 0x8010001 from environment

Starting kernel ...



U-Boot 2017.07-DEVEL (Apr 01 2024 - 14:50:01 +0000)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
Net:   MAC0 addr:e0:cb:bc:11:22:33
PHY ID1: 0x4d
PHY ID2: 0xd0b1
ipq40xx_ess_sw_init done
eth0
Autoboot in 5 seconds
FUZZY CRICKET #

We could proceed by putting OpenWrt directly in part.safe however if we ever had a bad flash there would be no way to recover except by manually reflashing NAND. Chain loading a u-boot build does delay booting by a few seconds, but offers a way to recover from a bad flash with just UART and tftpboot.

With the modified u-boot on flash, we can now tftpboot the OpenWrt initramfs and install it to flash with sysupgrade.


“Meraki Go offers simple networks for serious business.”

The Meraki Go GX20 uses the same PCB as the Z3, but without the WiFi radios populated (the lack of WiFi means you won’t find the GX20 in the FCC database). So, does the above technique work for the Meraki Go GX20?

U-Boot 2012.07-g03cdfe19e00f [local,local] (Aug 29 2017 - 11:59:45)

DRAM:  498 MiB
machid : 0x8010001
ERROR: Unknown board
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

In:    serial
Out:   serial
Err:   serial
machid: 8010001
Hit any key to stop autoboot:  0
Creating 1 MTD partitions on "nand0":
0x000000c00000-0x000007f00000 : "mtd=0"
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    126976 bytes
UBI: smallest flash I/O unit:    2048
UBI: VID header offset:          2048 (aligned 2048)
UBI: data offset:                4096
UBI: attached mtd1 to ubi0
UBI: MTD device name:            "mtd=0"
UBI: MTD device size:            115 MiB
UBI: number of good PEBs:        920
UBI: number of bad PEBs:         0
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    4096
UBI: number of internal volumes: 1
UBI: number of user volumes:     7
UBI: available PEBs:             364
UBI: total number of reserved PEBs: 556
UBI: number of PEBs reserved for bad PEB handling: 9
UBI: max/mean erase counter: 2/0
Read 0 bytes from volume part.safe to 84000000
No size specified -> Using max size (3284992)
## Booting kernel from FIT Image at 84000000 ...
   Using 'config@1' configuration
   Verifying Hash Integrity ... sha384,secp384r1:wired-arm-qca-RT-SECP384R1_1-rel+ OK
   Trying 'kernel@1' kernel subimage

No, unfortunately for u-boot 2012.07 which shipped on the Meraki Go GX20 I purchased, changing the device major results in ERROR: Unknown board and we can see from the sha384,secp384r1:wired-arm-qca-RT-SECP384R1_1-rel+ OK output that signature checking is still enabled.

However, the Meraki Go GX20 is a very similar device to the Z3. Is it possible that there is another way?

Examining the u-boot region of the Meraki Go GX20 with strings reveals the following:

California1
San Francisco
Cisco Meraki1%0#
fuzzy-cricket attestation ca1
Product Security0
170728184745Z
170827184745Z0

So the u-boot binary on the Meraki Go GX20 is signed with the same signing cert as u-boot on the Z3. What if we replace u-boot on the Meraki Go GX20 with the signed u-boot from the Z3 dump?

U-Boot 2017.07-RELEASE-g39cabb9bf3 (May 24 2018 - 14:07:32 -0700)

DRAM:  242 MiB
machid : 0x8010001
Product: meraki_Stinkbug
NAND:  ONFI device found
ID = 1d80f101
Vendor = 1
Device = f1
128 MiB
Using default environment

Bingo. And since the Z3 2017.07 u-boot release contains the secure boot downgrade bug, we can exploit it on the Meraki Go GX20 as well.

I am curious to see how u-boot 2012.07 on the Meraki Go GX20 implements signature validation, since unlike u-boot 2017.07 changing the device major does not disable signature verification of the payload.

Unfortunately, this is not possible. At the time of writing it has been 16 months since I requested the GPL source code for the Meraki Go GX20 and Meraki have yet to provide the u-boot source code for the device.


The EU and other jurisdictions claim to take the right to repair and e-waste seriously, but their actions thus far have ignored the elephant in the room. Many of the devices being sold in the past 5 years come with secure boot enabled, and thus are locked to running the OEM’s software.

In the case of the Meraki prodcuts, you have purchased the device and Meraki are selling a license only for the cloud management and hardware replacement service. But, if you stop paying Meraki, buy a “claimed” second-hand device, or Meraki discontinue support for the device you now have a worthless brick.

“Cisco Meraki may find it necessary to discontinue products for a number of reasons, including product line enhancements, market demand, technology innovation, or if the product simply matures over time and needs to be replaced by something functionally richer.”

To be clear, you are not leasing the hardware, and this is not “Hardware as a Service” you have bought the device in question and own it.

There are hundreds of SMD resistors on the device’s PCB. Any one of them could be tied to a GPIO which is polled by the SoC BootROM to enable or disable secure boot. Incorporating this into the hardware design would cost nothing, and would allow consumers the choice to re-use devices that were resold without proper decommissioning, from companies that were liquidated, from devices that were recycled as e-waste, or from devices where the manufacturer has decided to “maximize shareholder value” and end support. Physically opening the device and adding or removing a component to disable secure boot does not compromise user security in any way.

Routers like the Z3 and Meraki Go GX20 may appear to the untrained eye to be highly specialized devices, but in reality they are very similar to a Raspberry Pi with additional network interfaces, e.g. they are general purpose embedded computers.

It is past time that regulators start considering the user-harmful practices of companies in enforcing secure boot on these devices with no way for consumers to exercise their rights. This anti-consumer behaviour increases costs and creates more e-waste, solely for the benefit of the bottom line of companies that are (often) not even based in the same economic area. We should not, and cannot, accept such a status quo. There is no reason for these devices to become e-waste!


Impact: Device owners with physical access to their device and the appropriate hardware flashing tools can install a custom firmware.

Could this be accomplished remotely? No, not unless it is chained with another exploit providing Remote Code Execution (RCE) on the device. Anyone with an RCE on Meraki devices would not waste it on flashing a FOSS firmware, they would use it to build a botnet.

Can Meraki patch this? Yes, it is trivial for them to add a check in u-boot 2017.07 whether secure boot is enabled and enforce signature verification. Patching this is left as an exercise to the reader.

Will they patch it? Meraki announced the EoL of the Z3 in March 2024. They have previously been known to update u-boot to brick devices when users attempt to flash a custom firmware. It remains to be seen if they will continue this anti-consumer practice on the Z3 and Meraki Go GX20.

Responsible disclosure: Yes, this has been responsibly disclosed to you, the device owner.

Build and package your own software for OpenWRT

Today I am going to discuss how to build and package your own software for OpenWRT.

When I say “your own software” in this case I am referring to a C program which you want to cross-compile for the target SoC and install using the opkg package manager included in OpenWRT.

The program I wrote is a little more complicated than your standard “Hello World” application. Here’s what I wanted to do:
1) use libconfig to read a configuration file in /etc/config/ and then perform actions based on the configuration described in this file
2) use sqlite3 to create a database
3) write some meaningful data to the database

Here’s the program flow:
1) Open /etc/config/example-sqlite and read the values into variables
2) Open (or create) a new SQLite3 database file at the location defined in the above configuration file
3) Determine if the SQLite file is initialized with the target table we want to write to, and if not, create the table
4) Write the system load average to the database
5) Quit

To recap, this program is different from “Hello World” in the following ways:
1) It must read and understand a configuration file in libconfig syntax; this requires linking against the libconfig library, which we must tell opkg is a dependency
2) It must create or open an SQLite 3 database; this requires linking against the sqlite3 library, which we must tell opkg is a depenedency
3) It must perform some useful operations on this SQLite file

Let’s start with compiling the C file on your native architecture. Sure, you can just use cc/gcc from bash, but this isn’t any good to OpenWRT SDK, which expects that each package will have a makefile which can be used to compile the software.

load2sqlite.c

#include <sys/types.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <sqlite3.h>
#include <libconfig.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>

int main(int argc, char *argv[]) {
// ...

Most importantly above, we are including sqlite3.h for SQLite support, libconfig.h, and sys/stat.h, fcntl.h,errno.h to check if the SQLite3 database file exists or not.

You can compile this by hand quite easily, just by doing:
cc load2sqlite.c -lsqlite3 -lconfig -o load2sqlite

Okay, but how do we make this ready for OpenWRT SDK? By writing a makefile!

makefile

PROFILE = -O2 -s
CFLAGS = $(PROFILE)
LDFLAGS = -lsqlite3 -lconfig

all: main

# build it
main:
	$(CC) $(CFLAGS) load2sqlite.c $(LDFLAGS) -o load2sqlite

# clean it
clean:
	rm load2sqlite

Okay, so now if you type make in the directory, magically you will end up with an executable called load2sqlite!

But, this is a native binary, and it’s somewhat unlikely that your OpenWRT device is on the same architecture.

load2sqlite: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=9661b88e92b553d0556cbeeafccf04d2526c770f, stripped

If you run it, you’ll see that it looks for the sqlite database file, can’t find it, and so initalizes a new one with the “readings” table.

[hmartin@localhost src]$ ./load2sqlite 
Database file /tmp/sqlite3.db does not exist
Initialized database with readings table
[hmartin@localhost src]$ echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
2015-10-28 22:48:42|0.57|0.56|0.57

And if you run it again, without removing the SQLite3 file that was created, you’ll see this output:

[hmartin@localhost src]$ ./load2sqlite
SQLite database opened
Found readings table
[hmartin@localhost src]$ echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
2015-10-28 22:48:42|0.57|0.56|0.57
2015-10-28 22:49:00|0.47|0.54|0.57

Before we proceed further, I want to show you the directory structure so you have an idea of where we just were when we did this compilation. We are currently in the the src directory.

load2sqlite/
|-- Makefile
|-- README
`-- src
    |-- load2sqlite.c
    |-- load2sqlite.conf
    `-- makefile

Now let’s move up to the load2sqlite directory and work on the OpenWRT Makefile (seen above).

Here is the complete file, and then we will discuss it section by section:
Makefile

#
# Copyright (C) 2006-2015 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#

include $(TOPDIR)/rules.mk

PKG_NAME:=load2sqlite
PKG_VERSION:=1.0.1
PKG_RELEASE:=5
PKG_MAINTAINER:=Hal Martin 
PKG_LICENSE:=GPL-2
PKG_CONFIG_DEPENDS:=libsqlite3 libconfig

include $(INCLUDE_DIR)/package.mk

PKG_BUILD_DIR := $(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)

TARGET_LDFLAGS+= \
  -Wl,-rpath-link=$(STAGING_DIR)/usr/lib \
  -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/libconfig/lib \
  -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/sqlite/lib

define Package/load2sqlite
  SECTION:=utils
  CATEGORY:=Utilities
  DEPENDS:=+libsqlite3 +libconfig
  TITLE:=SQLite example program, creates or opens a user defined SQLite database
  URL:=https://github.com/halmartin/load2sqlite
  MENU:=1
endef

define Package/load2sqlite/description
 Example SQLite is a sample program built using libsqlite3 and libconfig
 which creates or opens a user-defined SQLite3 database and performs some
 simple verification checks on the file to ensure that the target table (readings)
 exists, and if not creates the table, then inserts a row with the current system
 time, and the load (1 minute, 5 minute, 15 minute).
endef

define Build/Prepare
	mkdir -p $(PKG_BUILD_DIR)
	$(CP) ./src/* $(PKG_BUILD_DIR)/
endef

define Build/Configure
endef

define Build/Compile
	$(MAKE) -C $(PKG_BUILD_DIR) $(TARGET_CONFIGURE_OPTS)
endef

define Package/load2sqlite/install
	$(INSTALL_DIR) $(1)/bin
	$(INSTALL_BIN) $(PKG_BUILD_DIR)/load2sqlite $(1)/bin/
	$(INSTALL_DIR) $(1)/etc/config
	$(INSTALL_CONF) $(PKG_BUILD_DIR)/load2sqlite.conf $(1)/etc/config/load2sqlite
endef

$(eval $(call BuildPackage,load2sqlite))

If you clone the OpenWRT source and take a look at basically any package, you’ll see a Makefile that looks similar to the one above.

Let’s look at the package information:

PKG_NAME:=load2sqlite
PKG_VERSION:=1.0.1
PKG_RELEASE:=5
PKG_MAINTAINER:=Hal Martin 
PKG_LICENSE:=GPL-2

Here is where we define core details of our package, such as the name (e.g. what opkg will know it as), the version (useful for upgrading later), maintainer, and license.

TARGET_LDFLAGS+= \
  -Wl,-rpath-link=$(STAGING_DIR)/usr/lib \
  -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/libconfig/lib \
  -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/sqlite/lib

Since we want to build a program which links against external libraries, we must also tell the compiler where to find the header files for these libraries, so that the linking process does not fail during compilation. Above you can see that we are linking to libconfig and sqlite libraries.

define Package/load2sqlite
  SECTION:=utils
  CATEGORY:=Utilities
  DEPENDS:=+libsqlite3 +libconfig
  TITLE:=SQLite example program, creates or opens a user defined SQLite database
  URL:=https://github.com/halmartin/load2sqlite
  MENU:=1
endef

This is where you define the package for the OpenWRT build system and declare things like dependencies, and the description that will be present when you run menuconfig (which is how you will select your package to be built as part of an image).

Without declaring dependencies, you may find that you can build, package, and install your software, but it won’t run! So, by declaring the dependencies (packages which provide the libraries we link against) we ensure that when we type opkg install load2sqlite and libconfig and libsqlite3 are not installed, opkg knows to go and install them before installing our program. Now we can safely run the program because all the required libraries are installed on the device!

define Build/Prepare
	mkdir -p $(PKG_BUILD_DIR)
	$(CP) ./src/* $(PKG_BUILD_DIR)/
endef

define Build/Configure
endef

Since our utility is quite simple, as *NIX software goes, the preparation steps are to create the build directory and copy the source from the source directory to the build directory. Since there is nothing to configure in our sample program, the configure step is empty (otherwise the OpenWRT build system will attempt to configure the package and fail because we haven’t bothered to implement this).

define Build/Compile
	$(MAKE) -C $(PKG_BUILD_DIR) $(TARGET_CONFIGURE_OPTS)
endef

define Package/load2sqlite/install
	$(INSTALL_DIR) $(1)/bin
	$(INSTALL_BIN) $(PKG_BUILD_DIR)/load2sqlite $(1)/bin/
	$(INSTALL_DIR) $(1)/etc/config
	$(INSTALL_CONF) $(PKG_BUILD_DIR)/load2sqlite.conf $(1)/etc/config/load2sqlite
endef

Finally, compile and install the software. As you can see above, I didn’t include an install directive in the makefile of the application, it is instead done manually within the OpenWRT Makefile. This is your choice, since I was designing this program specifically to run on OpenWRT, I saw no need to incorporate the installation steps in the makefile of the program.

And, finally:

$(eval $(call BuildPackage,load2sqlite))

This line is required for OpenWRT to build the package. Forget this line, and you will sit there wondering why your package is not being built!


Okay, now we have prepared our software to be built for OpenWRT. It would be stupid of me to get this far and not tell you how to compile it using the OpenWRT toolchain!

Following the excellent OpenWRT documentation, we need to set up a buildroot.

Install the dependencies (instructions for Debian/Ubuntu):

sudo apt-get install git-core build-essential libssl-dev libncurses5-dev unzip subversion mercurial

Clone the OpenWRT Chaos Calmer release:

git clone git://git.openwrt.org/15.05/openwrt.git

I find that the stock OpenWRT repository is a bit light on some of the software I like to have on my routers, so I take step 3 and install the additional feeds:

cd openwrt
./scripts/feeds update -a
./scripts/feeds install -a

Follow step 4 to ensure you have all the required dependencies installed on your host system!

make defconfig
make prereq
# don't forget to copy load2sqlite to package/utils/ before running this step, or the package won't appear in the menu!
make menuconfig

If everything has gone well thus far (e.g. no errors in the OpenWRT Makefile, and you put load2sqlite in package/utils/ then you should see the following in your menuconfig:

menuconfig_load2sqlite

menuconfig_load2sqlite_desc

Now I already have an official OpenWRT build installed on my router, so I don’t need to build an entire image, just the package I want to install. To do this, we must first build the cross compilation toolchain required to compile for a different CPU architecture.

Warning: the OpenWRT buildroot is around 6GB on disk, so ensure you have the necessary space before starting!

make tools/install
# this will take a while the first time
make toolchain/install
# this will also take a while the first time

When we have the tools and toolchain compiled, we can compile our package:

make package/load2sqlite/compile

This will create an ipkg file in bin/ramips/packages/base/load2sqlite_1.0.1-5_ramips_24kec.ipk which we need to copy to our router to install:

scp bin/ramips/packages/base/load2sqlite_1.0.1-5_ramips_24kec.ipk [email protected]:/tmp/
# scp completes
ssh [email protected]
[email protected]'s password:

BusyBox v1.23.2 (2015-07-25 03:03:02 CEST) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 CHAOS CALMER (15.05, r46767)
 -----------------------------------------------------
  * 1 1/2 oz Gin            Shake with a glassful
  * 1/4 oz Triple Sec       of broken ice and pour
  * 3/4 oz Lime Juice       unstrained into a goblet.
  * 1 1/2 oz Orange Juice
  * 1 tsp. Grenadine Syrup
 -----------------------------------------------------
root@OpenWrt:~# opkg install /tmp/load2sqlite_1.0.1-5_ramips_24kec.ipk 
Installing load2sqlite (1.0.1-4) to root...
Installing libsqlite3 (3081101-1) to root...
Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/packages/libsqlite3_3081101-1_ramips_24kec.ipk.
Installing libpthread (0.9.33.2-1) to root...
Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/base/libpthread_0.9.33.2-1_ramips_24kec.ipk.
Installing libconfig (1.4.9-1) to root...
Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/base/libconfig_1.4.9-1_ramips_24kec.ipk.
Configuring libpthread.
Configuring libconfig.
Configuring libsqlite3.
Configuring load2sqlite.

Now that our package is installed, we can test it!

root@OpenWrt:~# /bin/load2sqlite 
Database file /tmp/sqlite3.db does not exist
Initialized database with readings table

If you install sqlite3-cli we can inspect the row added to the file:

root@OpenWrt:~# opkg install sqlite3-cli
root@OpenWrt:~# echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
2015-10-31 20:47:33|0.76|0.4|0.25

Since this is just an example program, it is one-shot (e.g. not a daemon). If you really do want to track the load of our OpenWRT router, just add /bin/load2sqlite to crontab (e.g. every hour) and you’ll have this tracking info in the SQLite database.

If you run it multiple times, you get another row added to the file each time the program is run:

root@OpenWrt:~# /bin/load2sqlite 
SQLite database opened
Found readings table
root@OpenWrt:~# echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
2015-10-31 20:47:33|0.76|0.4|0.25
2015-10-31 20:53:43|0.02|0.2|0.22
2015-10-31 21:23:19|0.08|0.04|0.05

Note that by default the file is saved to /tmp/, which on OpenWRT is a ramdisk. This means that the file will be lost when you reboot, or if you leave it running unattended for too long, the file size will grow to the point where the ramdisk will consume all available memory and the router will crash. For this reason, I suggest you modify the configuration file /etc/config/load2sqlite to point to non-volatile storage (such as a USB stick).


Source code: https://github.com/halmartin/load2sqlite


Why write another OpenWRT software guide?

Well, while I was looking for resources on how to build and package software for OpenWRT, I ran into a lot of posts about people compiling simple “Hello World” programs for OpenWRT, but for my particular use case, I wanted to utilize multiple libraries in my program, and I couldn’t find any good instructions on how to compile a program with linked libraries for OpenWRT.

Disclaimer: I’m not a C expert, so maybe there are some headers there which are not strictly necessary, but it works for me and the executable size is quite small.

If you wish to further reduce the size of your executable, you can tell the compiler to strip it of the symbol table and relocation information. Do this by appending -s to the PROFILE line in the makefile. When I did this on my laptop, the output went from 9.9KB to 7.0KB, or a savings of 30%

I have tested this on Chaos Calmer (15.05), and I expect the instructions would also work on Barrier Breaker (14.07) however I didn’t try this, so I cannot say certainly that it will work.

Direct wifi traffic through a VPN with openwrt

I think we probably all know someone who’s received a copyright violation notice. Usually these notices list the user’s IP address, date, and copyrighted file that was shared while demanding some payment or the content owner will take the user to court.

Today we will explore how to setup a wireless access point that automatically tunnels traffic through a VPN, so that you don’t have to worry about the activities of your guests on your network.

Note: If your VPN provider does not push the “redirect-gateway” option then DNS queries from clients will still go through your normal internet connection. This means that activities on the guest wifi are not completely anonymous!

For this I will be using the following:

  • TP-Link WR703N running OpenWRT 12.09 (Attitude Adjustment)
  • 1GB USB key
  • a subscription-based VPN provider

I chose the WR703N mainly because I had one and it is small, has low power consumption, and is quite inexpensive.

There are many instructions for how to install OpenWRT on the WR703N, so I’m not going to discuss that here. Also the choice of VPN providers differs based on your needs and price range. I recommend reading the TorrentFreak articles on VPN providers to find out which one is best for you.

A 1GB USB key is required as the flash on the WR703N is not large enough to hold an OpenWRT installation with luci and openvpn installed. First we need to move the OpenWRT OS from the internal flash to the USB key, this will allow us to install the additional packages required, namely openvpn. I followed these instructions for how to transfer the OS from internal flash to USB. I’ll provide a tl;dr:

  • Partition the USB key with a DOS partition table, make at least one partition of type 82 (Linux)
  • Format this partition as ext4
  • Install block-mount, kmod-usb-core, kmod-usb-ohci, kmod-usb-storage, kmod-usb2, kmod-scsi-core, kmod-scsi-generic, kmod-fs-ext4, libblkid

Plug in the USB key. Check dmesg on the router and you should see that it recognizes the USB key as a block device. Create two temporary folders, one to mount the USB key at, and the other to bind mount /

mount /dev/sda1 /tmp/usb
mount --bind / /tmp/flash
tar -C /tmp/flash -cvf - . | tar -C /tmp/usb -xf -
umount /tmp/flash
umount /tmp/usb

Now that we’ve moved the OpenWRT installation to the USB key we have to configure the router to boot from the USB key instead of internal flash. Edit /etc/config/fstab and change the following:

config mount
     option device /dev/sda1
     option target /home
     option enabled 0

to:

config mount
     option device /dev/sda1
     option target /
     option enabled 1

Now reboot the router, it should boot off the USB key now.

Now there is lots of available space

Now there is lots of available space

I followed this post for most of the following openvpn configuration. Now go ahead and install the openvpn package:

opkg install openvpn

scp all the crt, ovpn and other openvpn configuration files to /etc/openvpn on the router.

You can test openvpn by ssh’ing into the router and running:

openvpn --config myconfig.ovpn

from /etc/openvpn. Assuming that works, now open the luci interface on the router to create a new interface:

  1. Go to the Network tab, click on Interfaces
  2. Create a new interface, I called mine “VPN” and set the protocol to “unmanaged”
  3. Specify tun0 as the network interface for the VPN interface
  4. Under “Advanced Settings” click the “Bring up on boot” checkbox

Now you have a choice, you can either:

  1. add the VPN interface we are in the process of creating above to the WAN zone, in which case a route with the prefix 0.0.0.0/1 will be added, which will supersede the WAN route of 0.0.0.0/0 through longest prefix matching, or
  2. create a firewall zone in luci to ensure that any traffic from LAN is automatically forwarded directly to the VPN, never going to the WAN. This is based on an openwrt wiki example

If you choose the second, then you need to do some additional work in luci:

  1. Go to the Network tab, click on Firewall
  2. Add a new Zone, I called mine “vpn” set it to Input:accept, Output:accept, Forward:accept
  3. Forward all traffic from the LAN to the vpn zone and visa versa, remove the WAN zone from the forwarding from the LAN zone.

Your zones should look like this now:

Firewall Zones

Firewall Zones

Go back to the Interface page and edit the VPN interface. Under the “Firewall Settings” tab change the zone from “wan” to “vpn”. The interface should look like this now:

VPN Interface Firewall Zone Settings

VPN Interface > Firewall Settings > Assign Firewall Zone “vpn”

There is an airvpn thread full of information on how to ensure that traffic goes from the LAN through the VPN. The above achieves something similar to the iptables rule mentioned in the airvpn thread.

Now that we have the routing all configured, you can go back to openvpn. If the ovpn file has “auth-user-pass” in it, you can create a text file which contains your VPN username on the first line, and your password on the second, and change the ovpn file to have “auth-user-pass credentials.txt” so openvpn will not prompt you for them when it connects.

Next we need to configure openvpn to start a boot:

  1. Go to the System tab, click on Startup
  2. At the bottom in the text box, add the following above “exit 0”
/usr/sbin/openvpn --cd /etc/openvpn --daemon --config /etc/openvpn/myvpn.ovpn &

Now we want to secure the router more. You might have some technically savvy guests who may try to break into the admin interface of your router to reconfigure it.

Before we block access to the management ports from the “bad” (guest-facing) side, we need to ensure that we don’t lock ourselves out of the router. Go to the Network tab, click on Firewall, click on “Port Forwards” add new rules to forward SSH (TCP 22), HTTP (TCP 80), and HTTPS (TCP 443) from WAN to the IP address of your router, in my case this is 192.168.1.1. Make absolutely sure you can access these ports from the WAN interface (your home LAN) before you do the following!

Now, go to the Network tab, click on Firewall, click on “Custom Rules” and add these rules in the space provided:

iptables -I zone_lan_ACCEPT 1 -p tcp -i wlan0 --dport 22 -j DROP
iptables -I zone_lan_ACCEPT 1 -p tcp -i wlan0 --dport 80 -j DROP
iptables -I zone_lan_ACCEPT 1 -p tcp -i wlan0 --dport 443 -j DROP

This blocks your wireless clients from accessing ports 22, 80, and 443 on the router, which means if they try to go to the luci interface or SSH into the router from the wireless side, they can’t! You need to restart the firewall for these changes to take effect.

The performance appears to be quite good. I am not sure precisely what the speed of my internet connection here is, but I was able to get over 6MBit/s down using the VPN and the speedof.me speed testing service, which seems very good.

That’s it. I recommend rebooting the router to make sure everything you did will survive a power cycle. IANAL but this solution should allow you to avoid any legal ramifications for the activities of guests on your IP address since they’ll be using a VPN and have a different termination IP address.

So, in summary:

  1. All traffic from wireless clients will be directed through the VPN, if the VPN is down wireless clients will not have internet, nor will they have access to your network
  2. Wireless clients are considered hostile, and as such are blocked from accessing ports 22, 80, and 443 on the router to prevent break-in attempts.