Category Archives: Linux

Meraki MX80: buildroot firmware

Ten years ago, before ARM was in everything, many embedded systems that did not justify using an x86 used PowerPC (or MIPS). This leads us to today’s subject: the Meraki MX80, an “enterprise security appliance.”

Meraki MX80 marketing image

Meraki MX80

The MX80 is End-of-Life (EOL) and can be purchased quite inexpensively on eBay.

The MX-series differs from other Meraki networking products of the 2010-era that we have previously covered (the MS220) in that it is PowerPC based (APM86290) with 2GB of RAM and 1GB of NAND flash. The factory bootlog of the device can be found in this GitHub gist.


MX80 PCB with uart header highlighted

MX80 UART header

Removing the cover allows you to connect to the UART header J3 (57600n8), with the pinout:

  1. VCC (don’t connect)
  2. Rx
  3. Tx
  4. GND

Ground (pin 4) is closest to the Ethernet ports.

Obtaining a root shell is very easy as u-boot has a 1 second boot delay and accepts input on UART. The default meraki_boot target sets the bootargs from meraki_bootargs which appends extra_bootargs, so just override rdinit with /bin/sh to prevent the Meraki OS from booting.

setenv extra_bootargs rdinit=/bin/sh
run meraki_boot

Once the MX80 has booted, bring up eth0 (port label Internet on the MX80):

$ mount -t proc proc /proc
$ mount -t sysfs sysfs /sys
$ ifconfig lo up
$ ifconfig eth0 up
$ udhcpc eth0

Once you have functional networking, you can dump the contents of NAND to a remote host for further analysis:

$ cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00100000 00020000 "firmware"
mtd1: 00100000 00020000 "environment"
mtd2: 00040000 00020000 "oops-old"
mtd3: 3fdc0000 00020000 "ubi"
mtd4: 40000000 00020000 "all"
mtd5: 0201d800 0001f800 "part1"
mtd6: 0201d800 0001f800 "part2"
mtd7: 0001f800 0001f800 "board-config"
mtd8: 2001f000 0001f800 "storage"
$ dd if=/dev/mtdblock4 bs=1M | gzip -c | nc -l -p 5000

Running binwalk on “part1” shows us the structure of Meraki’s firmware:

$ binwalk part1.bin

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
1024          0x400           device tree image (dtb)
16384         0x4000          device tree image (dtb)
131072        0x20000         uImage header, header size: 64 bytes, header CRC: 0xD84034B5, created: 2020-05-29 01:15:36, image size: 2447726 bytes, Data Address: 0x0, Entry Point: 0x0, data CRC: 0xCE3A4A5E, OS: Linux, CPU: PowerPC, image type: OS Kernel Image, compression type: gzip, image name: "Linux-3.4.113"
131136        0x20040         gzip compressed data, maximum compression, from Unix, last modified: 1970-01-01 00:00:00 (null date)
4194304       0x400000        uImage header, header size: 64 bytes, header CRC: 0xDA83B155, created: 2020-05-29 01:16:17, image size: 16719915 bytes, Data Address: 0x0, Entry Point: 0x0, data CRC: 0x2381914F, OS: Linux, CPU: PowerPC, image type: RAMDisk Image, compression type: lzma, image name: "Simple Ramdisk Image"
4194368       0x400040        LZMA compressed data, properties: 0x5D, dictionary size: 67108864 bytes, uncompressed size: -1 bytes

There is a device tree image at offset 0x400 which seems to be for the MX60 (codename bluestone). There is a second device tree image at offset 0x4000 for the MX80 (codename fullerene).

It is not as simple as creating a binary image with the DTB at offset 0x4000, the kernel at 0x200000, and initrd at 0x40000 because Meraki have modified u-boot to have a custom command meraki which reads a header, verifies the contents of the ubi partition part1 or part2 with SHA1, and then sets environment variables from addresses defined in the header.

The layout of the header is as follows:

Header field Data type (value)
SHA1_MAGIC uint32 (0x8e73ed8a)
HEADER_LEN int32
DATA_LEN int32
SHA1SUM char[20]
MERAKI_EXTRA_MAGIC uint32 (0xa1f0beef)
MERAKI_EXTRA_LEN uint16 (0x0006)
MERAKI_EXTRA_TYPE uint16 (0x0001)
IMAGE_OFFSET uint32 (0x20000)
RAMDISK_OFFSET uint32 (0x400000)
FDT_OFFSETS array uint32 (0x400 or 0x4000)

The FDT used to boot depends on the value of meraki_part_fdt_index in u-boot. For the MX80, the index of the FDT offset is 1, meaning the FDT located at 0x4000 is used to boot. The presence of two FDTs suggests that Meraki are using the same firmware for both the MX60 and MX80. Despite the MX80 being a dual core CPU only one CPU core is usable, there is no SMP support in the kernel provided by Meraki.


To simplify booting, I have written a post-image.sh script which generates the appropriate header and assembles a bootable firmware image as part of the buildroot build process. You can find instructions on how to build the firmware in the meraki-builder GitHub repository.

The 3.4 kernel provided by Meraki doesn’t have any of the features required by OpenWrt (e.g. overlayfs) and buildroot doesn’t have a package manager. If you just want something to boot and run SSH on, then the buildroot image fulfills that need. You will most probably want to customize buildroot to include the packages and configuration that suits your needs. Upstream support in OpenWrt is still a long way away, as the APM86290 does not have support in the mainline kernel.

Meraki MS220: PoE support

The last several posts in this series have focused primarily on getting a custom firmware running on the Meraki MS220-series switches, without much regard for preserving existing features. Since I am now at a point where my custom firmware is functional as a Layer 2-ish switch, my attention has turned to PoE, since many switches in the series have PoE support and that is feature I think switch owners (especially MS220-8P) are interested in.

From my investigation into libpoecore included in the Meraki firmware, PoE on the MS220-8P appears to be managed by Microsemi’s PD690xx series of Power over Ethernet Management chips (datasheet). The PD690xx series communicates over I2C with the management CPU to manage PoE on the switch ports (enable/disable PoE, set 802.3af/at modes, query power consumed by a PoE device).

We can confirm that the PD690xx communicates via I2C by running poe_server from Meraki’s firmware and enabling I2C tracing in the kernel:

# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 1358/1358   #P:1
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
      poe_server-682   [000] ....   560.356000: i2c_write: i2c-1 #0 a=030 f=0000 l=4 [13-32-0f-ff]
      poe_server-682   [000] ....   560.358000: i2c_result: i2c-1 n=1 ret=1
      poe_server-682   [000] ....   560.358000: i2c_write: i2c-1 #0 a=030 f=0000 l=2 [13-32]
      poe_server-682   [000] ....   560.358000: i2c_read: i2c-1 #1 a=030 f=0001 l=2
      poe_server-682   [000] ....   560.359000: i2c_reply: i2c-1 #1 a=030 f=0001 l=2 [0f-ff]
      poe_server-682   [000] ....   560.359000: i2c_result: i2c-1 n=2 ret=2
      poe_server-682   [000] ....   560.359000: i2c_write: i2c-1 #0 a=030 f=0000 l=4 [13-32-0f-ff]
      poe_server-682   [000] ....   560.360000: i2c_result: i2c-1 n=1 ret=1
      poe_server-682   [000] ....   560.360000: i2c_write: i2c-1 #0 a=030 f=0000 l=4 [13-9e-dc-03]
      poe_server-682   [000] ....   560.362000: i2c_result: i2c-1 n=1 ret=1

I2C tracing is extremely helpful, as running strace against poe_server directly will not yield useful output as to what operations it is performing to configure PoE.

While it is good news that we are able to recover the I2C commands via kernel tracing, it’s bad news in the sense that writing a new daemon to duplicate the features of poe_cli is non-trivial.


Thankfully, with the libpoecore from the Meraki firmware dump and free disassembly tools like Ghidra (sorry Hex-Rays, support MIPS in IDA Free ¯\_(ツ)_/¯), understanding some of the logic behind functionality provided by poe_server and poe_cli becomes much easier.

If you disassemble libpoecore, you can find the function hard_init which contains code to set up GPIO outputs. Interesting to note is that while the GPIO pins change depending on which switch ASIC is present, the sequence of GPIO outputs to configure the PD690xx remains constant.

Disassembler view of the function hard_init from the library libpoecore.so

The same GPIO configuration is executed when switch_brain is started (full strace output):

writev(1, [{iov_base="", iov_len=0}, {iov_base="echo 7 > /sys/class/gpio/export\n", iov_len=32}], 2echo 7 > /sys/class/gpio/export) = 32
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo 12 > /sys/class/gpio/export\n", iov_len=33}], 2echo 12 > /sys/class/gpio/export) = 33
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo out > /sys/class/gpio/gpio7/direction\n", iov_len=43}], 2echo out > /sys/class/gpio/gpio7/direction) = 43
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo out > /sys/class/gpio/gpio12/direction\n", iov_len=44}], 2echo out > /sys/class/gpio/gpio12/direction) = 44
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo 1 > /sys/class/gpio/gpio7/value\n", iov_len=37}], 2echo 1 > /sys/class/gpio/gpio7/value) = 37
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo 0 > /sys/class/gpio/gpio12/value\n", iov_len=38}], 2echo 0 > /sys/class/gpio/gpio12/value) = 38
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo 1 > /sys/class/gpio/gpio12/value\n", iov_len=38}], 2echo 1 > /sys/class/gpio/gpio12/value) = 38
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo 0 > /sys/class/gpio/gpio7/value\n", iov_len=37}], 2echo 0 > /sys/class/gpio/gpio7/value) = 37
writev(1, [{iov_base="", iov_len=0}, {iov_base="echo 1 > /sys/class/gpio/gpio12/value\n", iov_len=38}], 2echo 1 > /sys/class/gpio/gpio12/value) = 38

The datasheet for the luton26 ASIC used in the MS220-8P, MS220-24P, and MS22 (VDMS-10393) doesn’t list anything connected to GPIO 7, and GPIO 12 is used for either SFP17_SD or PHY7_LED1 depending on the overlay function chosen. The functionality of these GPIO pins is undefined in the ASIC datasheet, however libpoecore is setting them and manipulating their outputs.

We can implement the logic of hard_init in an init script to set up the GPIO pins in the same way, and the result is that the PD690xx is configured for auto mode. I am not sure how, there is nothing in the PD690xx datasheet which suggests GPIO pins can be used to configure the operating mode, but the switch will automatically negotiate and power a PoE device.

Writing a new daemon to communicate with the PD690xx will ultimately be necessary if fine control over PoE functionality is to be achieved. Without I2C communication to the PD690xx, it is not possible to query the power budget, or limit port power delivery. In the mean time, for those who do not mind unmanaged “plug-and-play” style, PoE can be considered functional.

MS220-8P: Custom firmware from scratch

To follow up on my previous posts about modifying the firmware for the Cisco Meraki MS220-8P, I have more progress to report.

Since I could not figure out how to fix serial output from userspace when booting from u-boot, I went back to booting with RedBoot and tried to focus on improving userspace from my first sloppy attempt to coerce the Meraki firmware into booting from NOR.

Here is the current situation:

  • RedBoot without CRC or kernel size checks
  • Kernel 3.18.123 with xz compression
  • Busybox userspace based on buildroot 2020.02.1
  • Meraki kernel modules: vtss_core, proclikefs, merakiclick, elts_meraki, and vc_click are successfully loaded during boot
  • The click router is successfully initialized, creating the interfaces arping, linux_mgmt, sw0_pcap, and wired0

However, networking is non-functional. The switch broadcasts DHCP requests, but no packets are ever passed from the switching ASIC to the management CPU:

arping    Link encap:Ethernet  HWaddr 88:15:44:73:22:31
          inet addr:169.254.55.143  Bcast:169.254.255.255  Mask:255.255.0.0
          inet6 addr: fe80::8a15:44ff:fe73:2231/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:33 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:5546 (5.4 KiB)

linux_mgmt Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.255.255.255
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

sw0_pcap  Link encap:Ethernet  HWaddr 00:01:02:03:04:05
          inet addr:169.254.83.119  Bcast:169.254.255.255  Mask:255.255.0.0
          inet6 addr: fe80::201:2ff:fe03:405/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:5476 (5.3 KiB)

wired0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

On the other side:

tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
06:02:32.423798 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::201:c0ff:fe1f:9fb2 > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff1f:9fb2 to_ex { }]
06:02:33.097129 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::201:c0ff:fe1f:9fb2 > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff1f:9fb2 to_ex { }]
06:03:20.969517 IP (tos 0x0, ttl 64, id 20247, offset 0, flags [none], proto UDP (17), length 332, bad cksum 2a8b (->1677)!)
    0.0.10.10.68 > 10.10.255.255.67: [bad udp cksum 0x17ca -> 0x03b6!] BOOTP/DHCP, Request from 88:15:44:21:65:10 (oui Unknown), length 304, xid 0xd3650d76, secs 123, Flags [none] (0x0000)
          Client-Ethernet-Address 88:15:44:21:65:10 (oui Unknown)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Client-ID Option 61, length 15: hardware-type 255, 44:21:65:10:00:03:00:01:88:15:44:21:65:10
            SLP-NA Option 80, length 0""
            NOAUTO Option 116, length 1: Y
            MSZ Option 57, length 2: 1472
            Hostname Option 12, length 13: "m881544216510"
            T145 Option 145, length 1: 1
            Parameter-Request Option 55, length 14:
              Subnet-Mask, Classless-Static-Route, Static-Route, Default-Gateway
              Domain-Name-Server, Hostname, Domain-Name, MTU
              BR, Lease-Time, Server-ID, RN
              RB, Option 119
            END Option 255, length 0
06:04:25.620540 IP (tos 0x0, ttl 64, id 13784, offset 0, flags [none], proto UDP (17), length 332, bad cksum 43ca (->2fb6)!)
    0.0.10.10.68 > 10.10.255.255.67: [bad udp cksum 0x178a -> 0x0376!] BOOTP/DHCP, Request from 88:15:44:21:65:10 (oui Unknown), length 304, xid 0xd3650d76, secs 187, Flags [none] (0x0000)
          Client-Ethernet-Address 88:15:44:21:65:10 (oui Unknown)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Client-ID Option 61, length 15: hardware-type 255, 44:21:65:10:00:03:00:01:88:15:44:21:65:10
            SLP-NA Option 80, length 0""
            NOAUTO Option 116, length 1: Y
            MSZ Option 57, length 2: 1472
            Hostname Option 12, length 13: "m881544216510"
            T145 Option 145, length 1: 1
            Parameter-Request Option 55, length 14:
              Subnet-Mask, Classless-Static-Route, Static-Route, Default-Gateway
              Domain-Name-Server, Hostname, Domain-Name, MTU
              BR, Lease-Time, Server-ID, RN
              RB, Option 119
            END Option 255, length 0

The primary reason for this appears to be because Meraki is using the click modular router. Click seems to be an attempt to lobotomize the Linux kernel networking stack for the sake of writing academic research papers. The kernel side of click was never upstreamed to mainline and is no longer maintained. Let us pour one out for the poor soul at Cisco who has to maintain this.

Since Meraki was spun out of MIT and the researchers who wrote Click were also at MIT, the relationship between Meraki and Click is clear. Finally Click found a practical use, in a now-obsolete product line from Cisco.

More reverse engineering is necessary to make Click functional enough to have basic connectivity to the management CPU.


The strace output of switch_brain is available in this gist. By first running switch_brain, and then overwriting the default traffic rules with “allow all” I was able to SSH to the management CPU:

/ # echo "allow all" > /click/nat/common_switch_nat/from_smc_filter/config
/ # echo "allow all" > /click/nat/common_switch_nat/from_mgmt_filter/config
/ # tail /tmp/messages | grep dropbear
Jan  1 00:15:49 buildroot authpriv.info dropbear[680]: Child connection from 10.10.10.1:39248
Jan  1 00:15:50 buildroot authpriv.notice dropbear[680]: Password auth succeeded for 'root' from 10.10.10.1:39248
/ # w
USER            TTY             IDLE    TIME             HOST
root            pts/0           00:00   Jan  1 00:15:50  10.10.10.1

I think it is now very clear that the “only” thing blocking full access to the management CPU is the Click configuration. My hope is to build the configuration from the switch_brain strace output and package that into an initscript.

I have updated the meraki-builder repository with the buildroot config file and overlay directory I use to inject init scripts (among other things).

If you would like to contribute, I have written some installation instructions for the current development firmware. If you find the correct voodoo of Click commands to access the management CPU, please open a PR 🥰 The voodoo has been found.