The real meaning of 10^9

I recently bought a 128GB USB mass storage device from Amazon here in Germany. The price I paid for this unbelievable amount of storage in my pocket? Just 22 Euros.

As you maybe know from a previous article, I’m never exactly pleased when I buy a storage device and find out that the manufacturer is actually selling less capacity than they’re advertising. Rather than turn this into another rant, I’d like to start a discussion with storage manufacturers about what 10^9 means.

First, some definitions:

Gigabyte: 1,000,000,000 bytes is the actual number of bytes in a “gigabyte” which has historically had the acronym “GB” shown.

And when you think about it, it makes sense. Recall your SI units, where you have kilo (1,000), mega (1,000,000), giga (1,000,000,000), tera (1,000,000,000,000), peta (1,000,000,000,000,000)… you get the point.

Gibibyte: 1,073,741,824 bytes. This is the number of bytes in “GiB” which is what computers typically operate in, because it’s a power of 2 (2^30).

So, why the confusion? Well, when you buy a computer with 8 “GB” of RAM, you’re actually buying a computer with 8 GiB of RAM. But, for historical reasons, it’s much more common to see electronics advertised with “GB” instead of “GiB” (although in recent years things seem to be changing, at least on the software side).

Some smart executive at a storage company long ago figured out that if they were advertising products as having 1 GB of capacity, that was actually 10^9 bytes, not 2^30 bytes, and they could increase profits if they started selling devices which were only 1,000,000,000 bytes instead of 1,073,741,824 bytes. That’s like 7% less storage!

https://commons.wikimedia.org/wiki/File:High_five!!.jpg

The celebration probably looked something like this

And in fact, storage manufacturers have been quite clear about this for a long time. They state quite clearly, in small print, on the back of the box or at the bottom of their product webpage, that the actual size of a Gigabyte is really 1,000,000,000 bytes. So, when you plug that shiny new storage device into your computer, and see 119.2GiB, well that’s just you failing to do the math to account for the difference between 10^9 and 2^30.

In fact, many manufacturers have support pages dedicated to outraged people who buy a device and haven’t read the fine print. Here is the SanDisk website explaining device capacity:
http://kb.sandisk.com/app/answers/detail/a_id/46/kw/capacity

I am not here trying to argue that electronics manufacturers should advertise storage devices advertised in GiB. They’re correctly advertising the capacity of the devices in Gigabytes, the SI unit. It would probably help SanDisk even more if on their website they used the appropriate definition for 2^30, which is Gibibyte. But as I said, consumers have gotten used to reading “GB” so when they see “GiB” they don’t understand what the difference is.

I know, I can hear you thinking right now “Okay, so what? Get to the point already, you’ve been droning on for over 400 words. Hey, are you paid by the word?”

First, I make no money from this website. And second, I said this was a discussion, not a rant. In a discussion, you must provide context and frame the problem. Otherwise it’s just blatent complaining.

Remember that SanDisk page (pictured above) where they helpfully explained the difference between a Gigabyte and Gibibyte for us? Well, I didn’t show you the whole page. Here is what is written directly after the screenshot pictured above explaining a gibibyte:
sandisk_capacity_explanation_revealed

So, basically what SanDisk is telling us here is that the actual size of the device isn’t even the advertised capacity multiplied by 10^9 bytes, it’s actually less. This is basically the legal equivalent of them saying “trust us, it really has 128,000,000,000 bytes inside, but you can’t use them all.”

Somehow, our governments have decided that this kind of advertising is legal.

I went back and looked at the Amazon.de page where I bought the product. It might shock you, dear reader, but there was no fine bullet point in the specifications saying “actual user storage less”:
amazon_de_128GB_cruzer_blade

This is understandable. If people saw “actual user storage less” mentioned in the product advertisement, they would probably be suspicious of the amount of storage they were actually buying, and sales would suffer.

I thought I would go look at other retailers to see if “actual user storage less” was mentioned anywhere on their websites. Here is the same product listed on Amazon.com:
amazon_com_128GB_cruzer_blade

What about NewEgg.com?
newegg_com_128GB_cruzer_blade

Okay, so the companies selling these devices aren’t overly eager to include this fine print, which SanDisk actually includes on their website. In small text, at the bottom of the page:
sandisk_com_128GB_cruzer_blade

I emailed SanDisk about this to ask why the “actual user storage less” wasn’t mentioned on any retailers website, and they responded:
sandisk_reply_1

While I disagree with their reply, I understand that retailers have some freedoms in how they advertise a product. However, with this in mind, I am sure that a small army of SanDisk lawyers would co-sign a cease and desist letter if I started advertising their products in any way which they determined was harming their brand value. But then something curious happened…

sandisk_reply_2

SanDisk claims that they don’t have any control over how retailers advertise their product, but then they state that these companies are “authorised distributors and resellers.”

Given the incredibly high percentage of counterfeit products being sold these days under the label of a well known brand, it’s clear that manufacturers need a trustworthy outlet to sell their goods, or consumers might begin to doubt the quality of their brand. That’s the economic impact of “electronics priacy” [PDF].

Managing your supply chain and maintaining your brand image costs companies millions, if not billions, of dollars every year. It’s serious business. People go to jail for importing and selling counterfeit products.

So, when a company claims that they have no control over how their product is advertised, I find that a bit difficult to believe. Legally they may not have an obligation to require retailers mention “actual user storage less” but morally and ethically they should ensure that their retailers do not advertise their product in a misleading way.

SanDisk is selling a “128GB” USB stick, which has a raw capacity of ~125GB (116GiB):
sandisk_cruzer_128GB_fdisk

They also mention in small text, on the back of the package, that “actual user storage less.” Too bad they didn’t state this anywhere on the actual retail page.

At this point, anyone who is sane will do the math and ask the question “You received 2.2% less capacity than was advertised. Why have you wasted your time writing about this?” and that’s because I’m scared of the precedent this is setting.

It’s true, it’s seemingly pointless to sit here and discuss the missing 2.2%. But 10 years ago, you would spend a lot of money to buy a USB stick which was 2GB. Even now, most people pay phone companies tens of dollars per month to transfer 3GB over 3G or LTE, or $90 per month if you’re unlucky enough to own a smartphone in Canada:
canadian_cell_plans

So even though as a percentage, it’s relatively small, it a not-insignificant amount of capacity that’s missing. If this was a 1TB device, you would be missing 23GB, and this is even before we get into Giga versus Gibi and formatting…

When I buy a device where the primary function is storage, I expect to be buying a device which can contain $CAPACITY * 10^9 bytes of data. This is important for things like data recovery. If I need to make an exact duplicate of the data on a storage device which is 128GB, I don’t want to have to worry about buying a device from a specific manufacturer, model, or serial number range to be able to store the data. This isn’t swapping the PCB on a broken hard drive, it’s just buying a simple storage device!

I’m scared of what message we’re sending to manufacturers when we allow them to sell us products with vague statements like “actual user storage less” and what it means for the future of the industry.

If I buy a smartphone which has 16GB of space, I expect to be able to use less, because the primary function of a smartphone is to be a pocket computer. I understand that capacity is required for the operating system, and that the actual capacity available to me will be less than the advertised amount.

But to buy a device whose sole purpose is to store data, and have that device provide less capacity than advertised. That’s scary, not for the 2.2% that I can’t use today, but because tomorrow it might be 5%, and in 10 years 20%.

Why is the capacity less? Perhaps they’re using NAND which doesn’t have space over provisioned for error correction and wear levelling, allowing them to fit a few more on a wafer. Or maybe the chips have bad regions which they’ve mapped around, meaning you get a slightly smaller capacity. I doubt we’ll ever get an explanation apart from “actual user storage less.”

SanDisk isn’t the only manufacturer doing this, but they’re certainly the worst offender that I’ve found. Recently I purchased some 16GB micro SDHC cards from Transcend, and they’re 0.5% under capacity as well. Luckily I have found that Samsung’s EVO line appears to at least provide the advertised capacity in 10^9 bytes. But, how much longer until everyone clues in to the get out of jail free card that is “actual user storage less” and starts selling devices under the 10^9 capacity?

The moral of the story here is: complaining on the internet is useless. Vote with your wallet. Return anything which is not actually 10^9 bytes of capacity, and rate it accordingly to warn other users. I’m not going to support a brand which, in my opinion, allows retailers to advertise their products in misleading ways. Besides, the performance sucked (4.5MB/s sequential write). You get what you pay for.

Linux 4.5 on a Bay Trail tablet

This post is a short update to my original article on booting Arch Linux on a Bay Trail tablet.

I originally wrote this for 4.4.5, but I wasn’t fast enough, and 4.5 was released before the post was completed, so might as well continue with a 4.5 kernel.

To simplify the build process I took the PKGBUILD for linux-mainline in AUR and modified it to build a mainline kernel with patches for SDIO WiFi on BayTrail.

If you’d like to build the kernel yourself (and you happen to run Arch Linux) you can download the PKGBUILD.

The firmware for the rtl8723bs card is in its own package, in keeping with the Arch Linux best practices for separating firmware from the kernel package. Download the firmware PKGBUILD.

Or, if you’d rather just have a newer kernel on your tablet which is already running Arch Linux, you can download the pre-built kernel package, and the firmware package.

I will be submitting both of these packages to AUR shortly.

Turns out you can actually get GRUB working with a menu if you build a standalone version of grub. However, the issue is that even though the grub menu works, there’s some issue with modesetting and you’ll never see any console after grub hands off to the kernel. You can download the standalone version of grub if you want to try, I wasn’t able to get any usable installer environment out of it. You can download standalone grub for ia32 (i686), you will also need grub.cfg.

$ tar -Jxf bootia32.tar.xz
$ cp bootia32.efi /mnt/archiso/EFI/boot/bootia32.efi
$ cp grub.cfg /mnt/archiso/EFI/boot/grub.cfg

Since grub video handoff isn’t working well, the only way I was able to successfully boot was to drop to command line by pressing c at the menu, and typing the following:

set root=hd0,msdos1
linux /arch/boot/x86_64/vmlinuz archisobasedir=arch archisolabel=ARCH_201603 video=VGA-1:[email protected]
initrd /arch/boot/x86_64/archiso.img
boot

There is a small issue with kernel oops, which has been present since at least 4.4.5:

[  164.281827] NMI watchdog: Watchdog detected hard LOCKUP on cpu 2
[  164.281913] Modules linked in:
[  164.281962]  intel_rapl intel_soc_dts_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw iTCO_wdt snd_soc_sst_bytcr_rt5640 iTCO_vendor_support hid_multitouch gf128mul glue_helper dcdbas ablk_helper cryptd pcspkr hci_uart snd_intel_sst_acpi mei_txe joydev input_leds snd_intel_sst_core btbcm snd_soc_rt5640 evdev snd_soc_sst_mfld_platform mousedev btintel mei lpc_ich mac_hid snd_soc_rl6231 thermal snd_soc_sst_match dw_dmac dw_dmac_core tpm_crb snd_soc_core bluetooth processor_thermal_device int3400_thermal int3403_thermal acpi_thermal_rel int3402_thermal i2c_hid int340x_thermal_zone snd_compress intel_soc_dts_iosf tpm_tis rfkill_gpio snd_pcm_dmaengine battery ac97_bus ac spi_pxa2xx_platform crc16 tpm snd_pcm i2c_designware_platform
[  164.283185]  acpi_pad i2c_designware_core 8250_dw snd_timer snd processor soundcore sch_fq_codel nfs lockd grace sunrpc fscache ip_tables x_tables overlay squashfs loop nls_iso8859_1 nls_cp437 vfat fat sd_mod uas usb_storage scsi_mod hid_generic usbhid hid i915 mmc_block button i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crc32c_intel xhci_pci drm xhci_hcd intel_gtt wmi serio video sdhci_acpi sdhci led_class r8723bs(O) cfg80211 rfkill mmc_core
[  164.283978] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G           O    4.5.0-byt #1
[  164.284073] Hardware name: Dell Inc. Venue 8 Pro 3845/XXXXXX, BIOS A02 12/29/2014
[  164.284169]  0000000000000086 9ad5a4512f59852f ffff880039d05b50 ffffffff812d25d1
[  164.284284]  0000000000000000 0000000000000000 ffff880039d05b68 ffffffff81116550
[  164.284399]  ffff880038ca8000 ffff880039d05ba0 ffffffff81156b4c 0000000000000001
[  164.284513] Call Trace:
[  164.284552]    [] dump_stack+0x63/0x82
[  164.284645]  [] watchdog_overflow_callback+0xe0/0xf0
[  164.284733]  [] __perf_event_overflow+0x8c/0x1d0
[  164.284815]  [] perf_event_overflow+0x14/0x20
[  164.284894]  [] intel_pmu_handle_irq+0x1e1/0x460
[  164.284980]  [] perf_event_nmi_handler+0x28/0x50
[  164.285062]  [] nmi_handle+0x5e/0x130
[  164.285133]  [] default_do_nmi+0x48/0x120
[  164.285207]  [] do_nmi+0xe2/0x130
[  164.285274]  [] end_repeat_nmi+0x1a/0x1e
[  164.285349]  [] ? poll_idle+0x39/0x80
[  164.285420]  [] ? poll_idle+0x39/0x80
[  164.285490]  [] ? poll_idle+0x39/0x80
[  164.285558]  <>  [] cpuidle_enter_state+0xf3/0x2f0
[  164.285655]  [] cpuidle_enter+0x17/0x20
[  164.285728]  [] call_cpuidle+0x2a/0x40
[  164.285800]  [] cpu_startup_entry+0x2c5/0x3a0
[  164.285878]  [] start_secondary+0x165/0x1a0
[  164.285964] INFO: NMI handler (perf_event_nmi_handler) took too long to run: 4.144 msecs
[  164.286069] perf interrupt took too long (32852 > 2495), lowering kernel.perf_event_max_sample_rate to 50100
[  172.707012] perf interrupt took too long (32619 > 4960), lowering kernel.perf_event_max_sample_rate to 25200

You’ll see a lot of these, however WiFi still continues to work, and the tablet didn’t kernel panic for me in the installer environment.

Hopefully someone finds this useful. I’ll have a write up on installing and using Arch Linux on the tablet in the coming weeks.

D-Link DAP-1520 hacking: Part 2

In Part 1 we looked at the hardware of the DAP-1520 and did some investigation into the stock D-Link firmware that runs on the device.

We found that there were two firmware images on the device, the main firmware (Image 1) and the recovery OS (Image 2) which is used when Image 1 fails verification.

Despite D-Link’s reputation for buggy firmwares, my infosec skills are still basic, and I wasn’t able to get telnetd running on the DAP-1520 to investigate the firmware more. Sure, we already have a dump of the firmware thanks to an SPI reader (and the update, available from D-Link’s website), but this only tells us what’s in the firmware, it doesn’t actually let us poke around at the hardware. Since my stated goal is to get OpenWrt running on the device, poking around at the hardware with a working OS is pretty important.

To accomplish this, I needed to build the firmware from the GPL source code published by D-Link. You can download the GPL source code for their routers from their Taiwanese website.

DAP-1520 GPL source code

DAP-1520 GPL source code

GPL source code for each firmware release

GPL source code for each firmware release

If you’re wondering why there’s a huge difference in the size of the source code between firmware versions, don’t. Firmware 1.05, despite the file extension, is just a tar file and is 185MB. Firmware 1.06 is a gzip compressed tar file and is 186MB. These inconsistencies are just the start of our wonderful journey with the D-Link source 😉

I’m building firmware 1.06, since newer is always better. I’ve just noticed that D-Link have published a file for firmware 1.07 on their Australian website. Hopefully they will release the GPL source for this firmware soon, I’m excited to see what vulnerabilities have been addressed in the web configurator!

Anyway, when you download the GPL source code, you will find that D-Link has included a README file, which describes how to build the firmware. This surprised me, I wasn’t expecting anything more than the source code, so some minor kudos go to D-Link for at least providing instructions.

Install & Build
===============
Environment: 
    1. Download Ubuntu 10.04.4 LTS from http://releases.ubuntu.com/lucid/
        http://releases.ubuntu.com/lucid/ubuntu-10.04.4-server-i386.iso
    2. Install Ubuntu 10.04.4 LTS server in your computer.
    3. Make sure your Ubuntu is 10.04.4 LTS(Lucid Lynx).
    4. Building image with ROOT privileges.	
	
Install:
    1.  Please update the list of available packages:
       ~#apt-get update
	   ~#apt-get install gcc build-essential zlib1g-dev bison flex subversion sharutils libncurses5-dev gawk help2man intltool pkg-config libglib2.0-dev	  
    2. Create a folder in root directory 
       ~#mkdir /tftpboot/
    3. Install the toolchain:
		~#cd /DAP-1520_A1_106b04_FOSS/toolchain
		A.	GCC
				1.	cp buildroot-gcc342.tar.bz2 /opt
				2.	tar jxvf buildroot-gcc342.tar.bz2
		B.	LZMA
			    1.  cp lzma-4.32.7.tar.gz /home/
			    2.  tar -xvf lzma-4.32.7.tar.gz
				3.  cd lzma-4.32.7
				4.	./configure
				5.	make
				6.	make install
			    7.  ldconfig	
		C.	XZ
				1.	cp xz-5.0.3.tar.bz2 /home/
				2.	tar jxvf xz-5.0.3.tar.bz2
				3.  cd xz-5.0.3
				4.	./configure
				5.	make
				6.	make install
		D.	mksquashfs
				1.  cp squashfs4.2.tar.bz2 /home/
				2.	tar jxvf squashfs4.2.tar.bz2
				3.	cd squashfs4.2/squashfs-tools
				4.	make
				5.	cp mksquashfs /opt/buildroot-gcc342/bin/mksquashfs_lzma-4.2
	4. Building the image & loader.
		(1). Please make sure the gcc-version is greater than 4.2.4
			 (You can type "~#gcc -v " to check the gcc-version)
		(2). Copy the DAP1520A1_GPL106b04.tar into /home/ directory
		     use following commands.
			~#tar -xvf DAP1520A1_GPL106b04.tar
		(3). You will get "AthSDK" directory.
			~#cd /home/AthSDK	
		(4). Into the AthSDK directory,and run following commands.
			(4-1). If you want to build normal image
			~#make clean
			~#make kernel_clean
			~#make 
			After make successfully, under "AthSDK/image/", you will get the normal image file "DAP1520A1_FW106B04.bin".
			(4-2). If you want to build backup image
			~#make -f Makefile.backup clean
			~#make kernel_clean
			~#make -f Makefile.backup 
			After make successfully, under "AthSDK/image/", you will get the backup image file "DAP1520A1_FW100B03.bin".
			(4-3). If you want to build loader
			~#make loader_clean
			~#make mtk_loader
			After make successfully, under "AthSDK/image/", you will get the loader file "DAP1520A1_FW100.boot".
	5. Update the new firmware by web interface provided by device.
	6. Congratulations! You got your specific image now.

Install Ubuntu 10.04? Thanks, I think I will pass. Ubuntu 10.04 isn't supported anymore, so good luck installing all the packages you need to support the build environment. So, instead I decided to build the firmware on my laptop, which runs a reasonably current version of Arch Linux. On Arch Linux /tmp is a ramdisk, so I just do all my work there and make symlinks when necessary. I wouldn't recommend using /tmp for work unless you have >8GB of RAM as the /tmp filesystem is by default 50% of your RAM, and the compiled source code is somewhere around 2GB give or take.

The first step is to decompress the toolchain and create a symlink from DAP-1520_A1_106b04_FOSS/toolchain/ to /opt/buildroot-gcc342 because a bunch of their makefiles are hard coded to look in this place for the toolchain.

$ cd $(mktemp -d)
$ tar -zxvf ~/Downloads/DAP-1520\ A1_ver1.06b04_FOSS.tar.gz
$ cd DAP-1520_A1_106b04_FOSS/toolchain
$ tar -jxf buildroot-gcc342.tar.bz2
$ tar -zxf lzma-4.32.7.tar.gz
$ tar -jxf squashfs4.2.tar.bz2
$ ln -s $(pwd)/buildroot-gcc342 /opt/buildroot-gcc342

Then you'll need to compile the versions of lzma and squashfs provided, for reasons which I will get into in a bit. Copy the lzma and mksquashfs_lzma-4.2 binaries into the bin folder of your toolchain. I don't recommend running make install as they do in the instructions, just run make and manually copy the binaries to the toolchain/bin directory.

$ cd lzma-4.32.7
$ ./configure
$ make
$ cp src/lzma/lzma ../buildroot-gcc342/bin/
$ cd ../squashfs4.2/squashfs-tools
$ make
$ cp mksquashfs ../../buildroot-gcc342/bin/mksquashfs_lzma-4.2

The copy of mksquashfs_lzma-4.2 included in the toolchain links against an ancient version of liblzma.so which has long since not existed in Arch Linux. Hence, it's easier just to compile the version from the source code included. Just install xz from your package manager, I didn't need to compile their specific version.

Now that we have "installed" the toolchain, we need to decompress the actual source code for the router firmware and "install" it in /home/AthSDK:

$ cd ../../../src/
$ tar -zxf DAP1520A1_GPL106b04.tar.gz
$ sudo ln -s $(pwd)/AthSDK /home/AthSDK

Now we are all set to start building it as per the D-Link instructions above:

$ cd AthSDK
$ make clean
$ make kernel_clean

At this point, we need to fix some of the source files or the compilation will fail. You will need to download and run the next few patches in the AthSDK directory or compilation will fail with errors.

$ wget https://watchmysys.com/blog/wp-content/uploads/2016/03/timeconst.patch
$ wget https://watchmysys.com/blog/wp-content/uploads/2016/03/busybox_makefile.patch
$ wget https://watchmysys.com/blog/wp-content/uploads/2016/03/busybox_features.patch
$ wget https://watchmysys.com/blog/wp-content/uploads/2016/03/timer_makefile.patch
$ wget https://watchmysys.com/blog/wp-content/uploads/2016/03/telnetd.patch
$ wget https://watchmysys.com/blog/wp-content/uploads/2016/03/nc.patch

Some explanation is in order:

  • Compiling the kernel will fail because Arch has a reasonably new verison of Perl, and the syntax in Perl >5.22 has changed since 2.6.36. You will need to apply the patch timeconst.patch to fix this.
  • Compiling busybox will fail because the syntax in the makefile is deprecated. You will need to apply the patch busybox_makefile.patch to fix this.
  • We want to have telnetd and nc in the image we create, for backdoors and stuff. You will need to apply the patch busybox_features.patch to enable these features.
  • D-Link includes an application called timer which expects an object file to compile, except this object file is never created. Removing the line fixes the error and as far as I know timer still works as intended. You will need to apply the patch timer_makefile.patch to fix this.
  • We need to create sysconfig scripts to start the telnetd and nc daemons on boot. The telnetd and nc patches create the sysconfig scripts in the rootfs folder of the D-Link OS
  • $ patch -p1 < timeconst.patch 
    patching file platform/MT7620/kernels/mips-linux-2.6.36.x/kernel/timeconst.pl
    $ patch -p1 < busybox_makefile.patch 
    patching file apps/busybox-1.6.1/Makefile
    $ patch -p1 < busybox_features.patch 
    patching file apps/busybox-1.6.1/.config
    $ patch -p1 < timer_makefile.patch   
    patching file apps/timer/Makefile
    $ sudo patch -p1 < telnetd.patch  
    patching file rootfs/target/etc/sysconfig/S3telnetd.sh
    $ sudo patch -p1 < nc.patch      
    patching file rootfs/target/etc/sysconfig/S4nc.sh
    $ chmod 755 rootfs/target/etc/sysconfig/*sh
    $ install -d rootfs/target/bin
    $ ln -s busybox_161 rootfs/target/bin/nc
    

    Now we can run the next command with sudo, as tar will attempt to create some files for the firmware which are owned by root. This will fail if not run as sudo:

    $ sudo make

    A long time later:

    
    =================== installing wireless ===================
    make -C wireless install || exit 1
    make[1]: Entering directory '/tmp/tmp.41xjcpqbrX/DAP-1520_A1_106b04_FOSS/src/AthSDK/wireless'
    make[1]: Leaving directory '/tmp/tmp.41xjcpqbrX/DAP-1520_A1_106b04_FOSS/src/AthSDK/wireless'
    =================== installing rootfs ===================
    make -C rootfs install || exit 1
    make[1]: Entering directory '/tmp/tmp.41xjcpqbrX/DAP-1520_A1_106b04_FOSS/src/AthSDK/rootfs'
    install -d /home/AthSDK/image
    rm -rf /home/AthSDK/rootfs/target/man/ /home/AthSDK/rootfs/target/lib/*.a
    Strip all .so
    find /home/AthSDK/rootfs/target/lib/ -name "*.so*" -exec mipsel-linux-uclibc-strip '{}' ';'
    cp -f /opt/buildroot-gcc342/lib/libdl* /home/AthSDK/rootfs/target/lib/
    mipsel-linux-uclibc-strip target/lib/libdl-0.9.30.so
    mipsel-linux-uclibc-strip: 'target/lib/libdl-0.9.30.so': No such file
    Strip all exec
    find /home/AthSDK/rootfs/target -type f -perm -u+x -exec mipsel-linux-uclibc-strip '{}' ';'
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/www/widget.cgi: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/www/tr069.cgi: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/www/save_configure.cgi: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/www/hnap.cgi: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/www/apply.cgi: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/www/library/test/success.html: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/usr/share/udhcpc/default.script: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/usr/share/udhcpc/default.bound-nodns: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/usr/share/udhcpc/default.bound-dns: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/lib/libavahi-core.la: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/lib/libavahi-common.la: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/lib/libexpat.la: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/lib/libdaemon.la: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/rdnssd-script: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/dhcp6c-script: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/host.conf: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/inittab: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/services: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/shadow: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/rdnssd/merge-hook: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/passwd: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/sysinfo: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/icon.ico: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/nvram.default: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/sysconfig/S2gpio.sh: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/issue: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/group: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/fstab: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/securetty: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/etc/rc.d/rcS: File format not recognized
    mipsel-linux-uclibc-strip: /home/AthSDK/rootfs/target/bin/gpio_event: File format not recognized
    Strip Atheros's *.ko
    find /home/AthSDK/rootfs/target/lib/modules/2.6.36.x -name "*.ko" -type f \
    	-exec mipsel-linux-uclibc-strip -g -S -d \
    	--strip-unneeded \
    	--remove-section=__kcrctab \
    	--remove-section=__kcrctab_gpl \
    	--remove-section=__param \
    	--remove-section=__ex_table \
    	--remove-section=__obsparm \
    	--remove-section=__versions \
    	--remove-section=.pdr \
    	--remove-section=.mdebug.abi32 \
    	--remove-section=.comment \
    	--remove-section=__ksymtab_gpl_future \
    	--remove-section=__kcrctab_gpl_future \
    	--remove-section=__ksymtab_unused \
    	--remove-section=__kcrctab_unused \
    	--remove-section=__ksymtab_unused_gpl \
    	--remove-section=__kcrctab_unused_gpl \
    	--remove-section=.ctors \
    	--remove-section=__markers \
    	--remove-section=__tracepoints \
    	--remove-section=_ftrace_events \
    	--remove-section=__mcount_loc \
    	-x '{}' ';'
    Strip Cameo's *.ko
    Remove unneeded files
    rm -f /home/AthSDK/rootfs/target/lib/modules/2.6.36.x/build
    rm -f /home/AthSDK/rootfs/target/lib/modules/2.6.36.x/modules.order
    rm -f /home/AthSDK/rootfs/target/lib/modules/2.6.36.x/source
    rm -rf /home/AthSDK/rootfs/target/include
    rm -rf /home/AthSDK/rootfs/target/lib/avahi
    rm -rf /home/AthSDK/rootfs/target/lib/pkgconfig
    rm -rf /home/AthSDK/rootfs/target/root
    rm -f /home/AthSDK/rootfs/target/lib/modules/2.6.36.x/net/ath_pktlog.ko
    cp /home/AthSDK/platform/MT7620/kernels/mips-linux-2.6.36.x/arch/mips/boot/vmlinux.* /home/AthSDK/image/ || exit 1;
    /home/AthSDK/tools/release_scripts/mkuImage.sh
    + case $BOARD_TYPE in
    + LDADDR=0x80000000
    ++ readelf -a /home/AthSDK/platform/MT7620/kernels/mips-linux-2.6.36.x/vmlinux
    ++ grep Entry
    ++ head -1
    ++ cut -d: -f 2
    + ENTRY='               0x8000c310'
    + /home/AthSDK/tools/release_scripts/mkimage -A mips -O linux -T kernel -C lzma -a 0x80000000 -e 0x8000c310 -n 'Linux Kernel Image' -d /home/AthSDK/image/vmlinux.lzma /home/AthSDK/image/vmlinux.lzma.ub
    Image Name:   Linux Kernel Image
    Created:      Sun Mar  6 23:20:20 2016
    Image Type:   MIPS Linux Kernel Image (lzma compressed)
    Data Size:    908125 Bytes = 886.84 kB = 0.87 MB
    Load Address: 0x80000000
    Entry Point:  0x8000C310
    /home/AthSDK/tools/release_scripts/release_rootfs.sh
    =================== Create SQUASHFS for DAP-1520 ===================
    Parallel mksquashfs: Using 4 processors
    Creating 4.0 filesystem on /home/AthSDK/image/MT7620-squash, block size 65536.
    [===========================================================================================\] 524/524 100%
    Exportable Squashfs 4.0 filesystem, xz compressed, data block size 65536
    	compressed data, compressed metadata, compressed fragments, compressed xattrs
    	duplicates are removed
    Filesystem size 3370.39 Kbytes (3.29 Mbytes)
    	28.91% of uncompressed filesystem size (11656.43 Kbytes)
    Inode table size 5286 bytes (5.16 Kbytes)
    	24.23% of uncompressed inode table size (21816 bytes)
    Directory table size 5948 bytes (5.81 Kbytes)
    	46.62% of uncompressed directory table size (12759 bytes)
    Number of duplicate files found 46
    Number of inodes 650
    Number of files 419
    Number of fragments 56
    Number of symbolic links  125
    Number of device nodes 53
    Number of fifo nodes 0
    Number of socket nodes 0
    Number of directories 53
    Number of ids (unique uids + gids) 1
    Number of uids 1
    	root (0)
    Number of gids 1
    	root (0)
    =================== MAX_ROOTFS_IMG_SIZE=3801062 Bytes =================== 
    0+1 records in
    1+0 records out
    3801062 bytes (3.8 MB) copied, 0.00318195 s, 1.2 GB/s
    =================== MT7620 Squashfs created for 8 MB FLASH ===================
    /home/AthSDK/tools/release_scripts/release_image.sh
    0+1 records in
    1+0 records out
    983040 bytes (983 kB) copied, 0.00105716 s, 930 MB/s
    make[1]: Leaving directory '/tmp/tmp.41xjcpqbrX/DAP-1520_A1_106b04_FOSS/src/AthSDK/rootfs'
    =================== Finish ===================
    

    We need to verify that the file produced matches the official firmware update available from D-Link. It really wouldn't do to flash an XZ compressed kernel when the bootloader expects an LZMA compressed kernel!

    $ binwalk image/DAP1520A1_FW106B04.bin 
    
    DECIMAL       HEXADECIMAL     DESCRIPTION
    --------------------------------------------------------------------------------
    0             0x0             uImage header, header size: 64 bytes, header CRC: 0x9D3D95E7, created: 2016-03-07 19:18:42, image size: 909160 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0x77D76472, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
    64            0x40            LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 2798288 bytes
    983040        0xF0000         Squashfs filesystem, little endian, version 4.0, compression:xz, size: 3451228 bytes, 650 inodes, blocksize: 65536 bytes, created: 2016-03-07 19:18:45

    Good, this matches the D-Link provided firmware update, so we should be all good to flash it to the router and see if we can login.

    Gotchas
    mksquashfs
    I had originally installed mksquashfs from pacman, because why would you want to use the old version included in the D-Link source?

    Hilariously, the version of mksquashfs_lzma-4.2 included in the D-Link source doesn't actually support LZMA compression at all, instead using xz compression by default!

    
    SYNTAX:/opt/buildroot-gcc342/bin/mksquashfs_lzma-4.2 source1 source2 ...  dest [options] [-e list of exclude
    dirs/files]
    
    Filesystem build options:
    -comp 		select  compression
    			Compressors available:
    				gzip
    				xz (default)
    
    (other options omitted)
    
    Compressors available and compressor specific options:
    	gzip (no options)
    	xz (default)
    	  -Xbcj filter1,filter2,...,filterN
    		Compress using filter1,filter2,...,filterN in turn
    		(in addition to no filter), and choose the best compression.
    		Available filters: x86, arm, armthumb, powerpc, sparc, ia64
    	  -Xdict-size 
    		Use  as the XZ dictionary size.  The dictionary size
    		can be specified as a percentage of the block size, or as an
    		absolute value.  The dictionary size must be less than or equal
    		to the block size and 8192 bytes or larger.  It must also be
    		storable in the xz header as either 2^n or as 2^n+2^(n+1).
    		Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K
    		etc.
    

    This differs from mksquashfs included in Arch Linux, which uses gzip by default! If you just go ahead and build the image using mksquashfs provided by your package manager, you will end up with a filesystem which is too large, and the build process will fail!

    
    $ mksquashfs -h
    SYNTAX:mksquashfs source1 source2 ...  dest [options] [-e list of exclude
    dirs/files]
    
    Filesystem build options:
    -comp 		select  compression
    			Compressors available:
    				gzip (default)
    
    (other options omitted)
    
    Compressors available and compressor specific options:
    	gzip (default)
    	  -Xcompression-level 
    		 should be 1 .. 9 (default 9)
    	  -Xwindow-size 
    		 should be 8 .. 15 (default 15)
    	  -Xstrategy strategy1,strategy2,...,strategyN
    		Compress using strategy1,strategy2,...,strategyN in turn
    		and choose the best compression.
    		Available strategies: default, filtered, huffman_only,
    		run_length_encoded and fixed
    	lzma (no options)
    	lzo
    	  -Xalgorithm 
    		Where  is one of:
    			lzo1x_1
    			lzo1x_1_11
    			lzo1x_1_12
    			lzo1x_1_15
    			lzo1x_999 (default)
    	  -Xcompression-level 
    		 should be 1 .. 9 (default 8)
    		Only applies to lzo1x_999 algorithm
    	lz4
    	  -Xhc
    		Compress using LZ4 High Compression
    	xz
    	  -Xbcj filter1,filter2,...,filterN
    		Compress using filter1,filter2,...,filterN in turn
    		(in addition to no filter), and choose the best compression.
    		Available filters: x86, arm, armthumb, powerpc, sparc, ia64
    	  -Xdict-size 
    		Use  as the XZ dictionary size.  The dictionary size
    		can be specified as a percentage of the block size, or as an
    		absolute value.  The dictionary size must be less than or equal
    		to the block size and 8192 bytes or larger.  It must also be
    		storable in the xz header as either 2^n or as 2^n+2^(n+1).
    		Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K
    		etc.
    

    At least the build process will fail and tell you the image is too large, instead of making an image which will brick your router...

    lzma
    Again I was wondering, why should I use the old version of LZMA included in the D-Link source, when Arch Linux ships which a much newer (and thus better) version of LZMA? It might occur to you, that I haven't learned my lesson yet from the previous experience with mksquashfs...

    The lzma included in D-Link's source:

    $ /opt/buildroot-gcc342/bin/lzma -h
    
    lzma 4.32.7 Copyright (C) 2005 Ville Koskinen
    Based on LZMA SDK 4.32 Copyright (C) 1999-2005 Igor Pavlov
    
    Usage: /opt/buildroot-gcc342/bin/lzma [flags and input files in any order]
      -c --stdout       output to standard output
      -d --decompress   force decompression
      -z --compress     force compression
      -k --keep         keep (don't delete) input files
      -f --force        force overwrite of output file and compress links
      -t --test         test compressed file integrity
      -S .suf  --suffix .suf   use suffix .suf on compressed files
      -q --quiet        suppress error messages
      -v --verbose      be verbose
      -h --help         print this message
      -L --license      display the license information
      -V --version      display version numbers of LZMA SDK and lzma
      -1 .. -2          fast compression
      -3 .. -9          good to excellent compression. -7 is the default.
         --fast         alias for -1
         --best         alias for -9 (usually *not* what you want)
    
      Memory usage depends a lot on the chosen compression mode -1 .. -9.
      See the man page lzma(1) for details.
    

    And lzma from Arch Linux:

    $ lzma -h
    Usage: lzma [OPTION]... [FILE]...
    Compress or decompress FILEs in the .xz format.
    
      -z, --compress      force compression
      -d, --decompress    force decompression
      -t, --test          test compressed file integrity
      -l, --list          list information about .xz files
      -k, --keep          keep (don't delete) input files
      -f, --force         force overwrite of output file and (de)compress links
      -c, --stdout        write to standard output and don't delete input files
      -0 ... -9           compression preset; default is 6; take compressor *and*
                          decompressor memory usage into account before using 7-9!
      -e, --extreme       try to improve compression ratio by using more CPU time;
                          does not affect decompressor memory requirements
      -T, --threads=NUM   use at most NUM threads; the default is 1; set to 0
                          to use as many threads as there are processor cores
      -q, --quiet         suppress warnings; specify twice to suppress errors too
      -v, --verbose       be verbose; specify twice for even more verbose
      -h, --help          display this short help and exit
      -H, --long-help     display the long help (lists also the advanced options)
      -V, --version       display the version number and exit
    
    With no FILE, or when FILE is -, read standard input.
    
    Report bugs to  (in English or Finnish).
    XZ Utils home page: 
    

    So, apart from the newer output looking much more like a standard GNU utility, you might have noticed that the older copy of lzma compresses with a default compression of -7 while the newer version compresses with a default compression of -6.

    If you think this doesn't make a difference, let me just tell you now, it does. A big one. The difference between -6 and -7 is the difference between a kernel that boots, and one that doesn't.

    This firmware was built with the D-Link SDK version of lzma and will boot:

    $ binwalk image/DAP1520A1_FW106B04.bin 
    
    DECIMAL       HEXADECIMAL     DESCRIPTION
    --------------------------------------------------------------------------------
    0             0x0             uImage header, header size: 64 bytes, header CRC: 0x46768407, created: 2016-03-05 22:33:15, image size: 909272 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0x4993B2D9, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
    64            0x40            LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 2798288 bytes
    983040        0xF0000         Squashfs filesystem, little endian, version 4.0, compression:xz, size: 3445784 bytes, 655 inodes, blocksize: 65536 bytes, created: 2016-03-05 22:33:20
    

    This firmware was built with the Arch Linux version of lzma and won't boot:

    $ binwalk image/DAP1520A1_FW106B04.bin 
    
    DECIMAL       HEXADECIMAL     DESCRIPTION
    --------------------------------------------------------------------------------
    0             0x0             uImage header, header size: 64 bytes, header CRC: 0x7A57558F, created: 2016-03-07 20:41:46, image size: 908147 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0x8E0F6C03, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
    983040        0xF0000         Squashfs filesystem, little endian, version 4.0, compression:xz, size: 3451428 bytes, 653 inodes, blocksize: 65536 bytes, created: 2016-03-07 20:41:49
    

    So, what exactly happens when you flash this image to the device? Does it do some verification before flashing and stop you? Does it flash the image, and then when Image 1 fails to boot it boots the linux4b image and start rootfsb so you can recover the device?

    Not quite...

    Image1 Try Counter --> 0
    
    Image1: OK Image2: OK
    Both images are OK!!!
    
    =================================================
    
    Please choose the operation: 
       1: Load system code to SDRAM via TFTP. 
       2: Load system code then write to Flash via TFTP. 
       3: Boot system code via Flash (default).
       4: Entr boot command line interface.
       7: Load Boot Loader code then write to Flash via Serial. 
       9: Load Boot Loader code then write to Flash via TFTP. 
     1  0 
       
    3: System Boot system code via Flash.
    ## Booting image at bc050000 ...
    raspi_read: from:50000 len:40 
       Image Name:   Linux Kernel Image
       Image Type:   MIPS Linux Kernel Image (lzma compressed)
       Data Size:    908132 Bytes = 886.8 kB
       Load Address: 80000000
       Entry Point:  8000c310
    raspi_read: from:50040 len:ddb64 
       Verifying Checksum ... OK
       Uncompressing Kernel Image ... LZMA ERROR 1 - must RESET board to recover
    

    So, no. No verification of the validity of the kernel in the update before flashing. And no, it won't boot from Image 2. You will just see this error, over and over, while the device resets. You might think that Image1 Try Counter would increment, and after a threshold it would boot into the recovery environment, but no. Congratulations, you are now the proud owner of a brick. Get out your SPI flashing tool, because there's no other way around this disaster.

    This does beg the question, how do you get the device to boot into Image 2? Well, after the LZMA compression snafu on the kernel, I thought I would save some time and just flash the vmlinuz.ub file created by the build script to flash and be done with it... nope!

    
    Check image validation:
    Image1 Header Magic Number --> OK
    Image2 Header Magic Number --> OK
    Image1 Header Checksum --> OK
    Image2 Header Checksum --> OK
    Image1 Data Checksum --> raspi_read: from:50040 len:de000 
    Failed
    Image2 Data Checksum --> raspi_read: from:4f0040 len:ca8f4 
    OK
    Image1 Stable Flag --> Not stable
    Image1 Try Counter --> 0
    
    Image1: Broken Image2: OK
    Only Image1 is borken!!
    
    =================================================
    
    Please choose the operation: 
       1: Load system code to SDRAM via TFTP. 
       2: Load system code then write to Flash via TFTP. 
       3: Boot system code via Flash (default).
       4: Entr boot command line interface.
       7: Load Boot Loader code then write to Flash via Serial. 
       9: Load Boot Loader code then write to Flash via TFTP. 
     1  0 
       
    3: System Boot system code via Flash.
    ## Booting image at bc4f0000 ...
    raspi_read: from:4f0000 len:40 
       Image Name:   Linux Kernel Image
       Image Type:   MIPS Linux Kernel Image (lzma compressed)
       Data Size:    829684 Bytes = 810.2 kB
       Load Address: 80000000
       Entry Point:  8000c310
    raspi_read: from:4f0040 len:ca8f4 
       Verifying Checksum ... OK
       Uncompressing Kernel Image ... OK
    No initrd
    ## Transferring control to Linux (at address 8000c310) ...
    ## Giving linux memsize in MB, 64
    
    Starting kernel ...
    
    
    LINUX started...
    
     THIS IS ASIC
    Linux version 2.6.36.x ([email protected]) (gcc version 3.4.2) #1 Thu Sep 26 16:47:12 CST 2013
    

    Couple of things to note here:

    1. Only Image1 is borken!!
    2. The kernel in Image 2 is much older, and was built on a different host, than the kernel in Image 1 from D-Link

    When you do flash a working firmware back onto the router, you get a surprise when it boots. Because Image1 is borken!! the device rewrites all the nvram variables to their defaults.

    
    init NVRAM_SPACE from mtdblock size
    init nvram memory map size: 0x10000 order of pages: 0x4
    nvram module init:
    	/dev/nvram major number 225 glues to mtd: "nvram" size: 0x00010000
    	nvram_space: 0x00010000 mapped via mmap(2)
    openfile :/etc/sysinfo
    openfile :/etc/nvram.default
    nvram_sanity_check: restore key: uplink_set_by_user="0"
    nvram_sanity_check: restore key: language="default"
    nvram_sanity_check: restore key: ap_ipv6_wan_specify_dns="0"
    nvram_sanity_check: restore key: ap_ipv6_autoconfig_secondary_dns=""
    nvram_sanity_check: restore key: ap_ipv6_autoconfig_primary_dns=""
    nvram_sanity_check: restore key: ap_ipv6_autoconfig_dns_enable="0"
    nvram_sanity_check: restore key: ap_ipv6_static_secondary_dns=""
    nvram_sanity_check: restore key: ap_ipv6_static_primary_dns=""
    nvram_sanity_check: restore key: ap_ipv6_static_default_gw=""
    nvram_sanity_check: restore key: ap_ipv6_static_prefix_length=""
    nvram_sanity_check: restore key: ap_ipv6_static_lan_ip=""
    nvram_sanity_check: restore key: ap_ipv6_wan_proto="ipv6_autoconfig"
    nvram_sanity_check: restore key: pure_support_url="http://support.dlink.com/products/view.asp?productid=DAP-1520"
    nvram_sanity_check: restore key: pure_reboot_page="reboot.htm"
    nvram_sanity_check: restore key: pure_parental_url=""
    nvram_sanity_check: restore key: pure_block_url=""
    nvram_sanity_check: restore key: pure_wireless_url_new="/Wireless.htm"
    nvram_sanity_check: restore key: pure_wireless_url="/wireless.htm"
    nvram_sanity_check: restore key: pure_presentation_url="/Device_Info.htm"
    nvram_sanity_check: restore key: pure_model_description="Wireless Repeater"
    nvram_sanity_check: restore key: pure_vendor_name="D-Link"
    nvram_sanity_check: restore key: pure_device_name="D-Link Systems DAP-1520"
    nvram_sanity_check: restore key: pure_type_new="WiFiAccessPoint"
    nvram_sanity_check: restore key: pure_type="Repeater"
    nvram_sanity_check: restore key: default_downlink_ssid="1"
    nvram_sanity_check: restore key: wlan1_wps_wizard="0"
    nvram_sanity_check: restore key: setup_wizard_ap="1"
    nvram_sanity_check: restore key: log_response_type="system|debug|attack|dropped|notice"
    nvram_sanity_check: restore key: log_current_page="0"
    nvram_sanity_check: restore key: log_total_page="0"
    nvram_sanity_check: restore key: log_per_page="10"
    nvram_sanity_check: restore key: log_notice="1"
    nvram_sanity_check: restore key: log_dropped_packets="0"
    nvram_sanity_check: restore key: log_attacks="1"
    nvram_sanity_check: restore key: log_debug_information="0"
    nvram_sanity_check: restore key: log_system_activity="1"
    nvram_sanity_check: restore key: syslog_server="0/0.0.0.0"
    nvram_sanity_check: restore key: time_daylight_offset="3600"
    nvram_sanity_check: restore key: time_daylight_saving_end_time="1"
    nvram_sanity_check: restore key: time_daylight_saving_end_day_of_week="1"
    nvram_sanity_check: restore key: time_daylight_saving_end_week="2"
    nvram_sanity_check: restore key: time_daylight_saving_end_month="11"
    nvram_sanity_check: restore key: time_daylight_saving_start_time="1"
    nvram_sanity_check: restore key: time_daylight_saving_start_day_of_week="1"
    nvram_sanity_check: restore key: time_daylight_saving_start_week="3"
    nvram_sanity_check: restore key: time_daylight_saving_start_month="3"
    nvram_sanity_check: restore key: time_daylight_saving_enable="0"
    nvram_sanity_check: restore key: ntp_sync_interval="168"
    nvram_sanity_check: restore key: ntp_default_server="ntp1.dlink.com,ntp.dlink.com.tw"
    nvram_sanity_check: restore key: ntp_server=""
    nvram_sanity_check: restore key: time_zone_area="4"
    nvram_sanity_check: restore key: time_zone="-128"
    nvram_sanity_check: restore key: ntp_client_enable="0"
    nvram_sanity_check: restore key: session_timeout="180"
    nvram_sanity_check: restore key: graph_enable="none"
    nvram_sanity_check: restore key: system_time="2011/01/01/00/00/00"
    nvram_sanity_check: restore key: serial_number="none"
    nvram_sanity_check: restore key: model_url="http://support.dlink.com"
    nvram_sanity_check: restore key: model_name="D-Link Repeater"
    nvram_sanity_check: restore key: manufacturer_url="http://www.dlink.com"
    nvram_sanity_check: restore key: manufacturer="D-Link"
    nvram_sanity_check: restore key: friendlyname="DAP-1520"
    nvram_sanity_check: restore key: model_number="DAP-1520"
    nvram_sanity_check: restore key: hostname="DAP-1520"
    nvram_sanity_check: restore key: wlan1_11n_protection="auto"
    nvram_sanity_check: restore key: wlan1_wps_enable="1"
    nvram_sanity_check: restore key: wlan1_psk_pass_phrase="1234567890"
    nvram_sanity_check: restore key: wlan1_psk_cipher_type="both"
    nvram_sanity_check: restore key: wlan1_wep_display="hex"
    nvram_sanity_check: restore key: wlan1_wep128_key="00000000000000000000000000"
    nvram_sanity_check: restore key: wlan1_wep64_key="0000000000"
    nvram_sanity_check: restore key: wlan1_security="disable"
    nvram_sanity_check: restore key: wlan1_ssid=""
    nvram_sanity_check: restore key: wlan_repeater_mode="1"
    nvram_sanity_check: restore key: wlan0_disable_wps_pin="1"
    nvram_sanity_check: restore key: wlan0_wps_configured_mode="5"
    nvram_sanity_check: restore key: wlan0_wps_enable="1"
    nvram_sanity_check: restore key: wlan0_disablecoext="0"
    nvram_sanity_check: restore key: wlan0_rxchainmask="3"
    nvram_sanity_check: restore key: wlan0_txchainmask="3"
    nvram_sanity_check: restore key: wlan0_gkey_rekey_time="3600"
    nvram_sanity_check: restore key: wlan0_11n_protection="auto"
    nvram_sanity_check: restore key: wlan0_wmm_enable="1"
    nvram_sanity_check: restore key: wlan0_short_gi="1"
    nvram_sanity_check: restore key: wlan0_partition="0"
    nvram_sanity_check: restore key: wlan0_dtim="1"
    nvram_sanity_check: restore key: wlan0_fragmentation="2346"
    nvram_sanity_check: Raeth v3.0 (reTaskletst,SkbRecycleo)
    nvram_sanity_check: restore key: wlan0_beacon_interval="100"
    nvram_sanity_check: restore key: wlan0_txpower="100"
    nvram_sanity_check: restore key: wlan0_psk_cipher_type="both"
    nvram_sanity_check: restore key: wlan0_wep_display="hex"
    nvram_sanity_check: restore key: wlan0_wep128_key="00000000000000000000000000"
    nvram_sanity_check: restore key: wlan0_wep64_key="0000000000"
    nvram_sanity_check: restore key: wlan0_ssid_broadcast="1"
    nvram_sanity_check: restore key: wlan0_dot11_mode="11bgn"
    nvram_sanity_check: restore key: wlan0_auto_channel_enable="1"
    nvram_sanity_check: restore key: wlan0_channel="6"
    nvram_sanity_check: restore key: wlan0_enable="1"
    nvram_sanity_check: restore key: wlan0_5g_fragmentation="2346"
    nvram_sanity_check: restore key: wlan0_5g_rts_threshold="2347"
    nvram_sanity_check: restore key: wlan0_5g_dfs_enable="0"
    nvram_sanity_check: restore key: wlan0_5g_11n_protection="adevice eth2 entered promiscuous mode
    uto"
    nvram_sanity_check: restore key: wlan0_5g_wep128_key="00000000000000000000000000"
    nvram_sanity_check: restore key: wlan0_5g_wep64_key="0000000000"
    nvram_sanity_check: restore key: wlan0_5g_psk_cipher_type="both"
    nvram_sanity_check: restore key: wlan0_5g_wep_display="hex"
    nvram_sanity_check: restore key: wlan0_5g_wmm_enable="1"
    nvram_sanity_check: restore key: wlan0_5g_txpower="100"
    nvram_sanity_check: restore key: wlan0_5g_dtim="1"
    nvram_sanity_check: restore key: wlan0_5g_beacon_interval="100"
    nvram_sanity_check: restore key: wlan0_5g_auto_channel_enable="1"
    nvram_sanity_check: restore key: wlan0_5g_channel="36"
    nvram_sanity_check: restore key: wlan0_5g_dot11_mode="11anac"
    nvram_sanity_check: restore key: dhcpc_enable="1"
    nvram_sanity_check: restore key: ap_device_name="dlinkap"
    nvram_sanity_check: restore key: ap_secondary_dns="0.0.0.0"
    nvram_sanity_check: restore key: ap_primary_dns="0.0.0.0"
    nvram_sanity_check: restore key: ap_gateway="0.0.0.0"
    nvram_sanity_check: restore key: ap_netmask="255.255.255.0"
    nvram_sanity_check: restore key: ap_ipaddr="192.168.0.50"
    nvram_sanity_check: restore key: lan_bridge="br0"
    nvram_sanity_check: restore key: lan_eth="eth2"
    nvram_sanity_check: restore key: admin_password=""
    nvram_sanity_check: restore key: admin_username="admin"
    

    Solid defaults there, D-Link. I think this is the first device I've ever encountered where the admin password was actually blank. I setup the device initially, and then it rebooted and asked me for a password to login. Even working in IT, I never thought to try an empty password. I mean, who does that?! D-Link does that.

    And just while writing this I realized that I can look at the nvram defaults any time I want to in /etc/nvram.defaults. This is why you use the 15 minute rule, people.

    Below is a firmware with nc running on port 8023 if you have a DAP-1520 and you want to poke around the D-Link firmware. Telnet asks for a username and password, and none of the combinations I could think of let me login.

    nc 192.168.0.50 8023

    Enjoy your root shell!

    Firmware DAP1520A1_FW106B04.bin (gzip compressed) with nc backdoor.
    md5sum: 31397369d0631183c3823d9933bede5f
    sha1sum: 2951f4e36b05014cbc327acf5c9d6e860ac2f0a5
    sha256sum: 6dd416c6f26e17f059dcc531e2a882a64e3af0594bd3720da669af631c34e50b

    D-Link DAP-1520 hacking: Part 1

    What do you do with a device you never would have bought for yourself, but received for free? Say welcome the D-Link DAP-1520, a “WiFi Extender” that was given to me by O2 as a bonus for signing up with them. Hopefully they aren’t expecting it back in one piece…

    So, what is the DAP-1520? Executive summary:

  • Supports 2.4GHz at 300MBps and 5GHz at 433MBps (thanks to SmallNetBuilder for demystifying this)
  • Repeats the packets from your existing WiFi network for extending range
  • Will also turn a 2.4GHz network into 5GHz through the repeating process (or vice versa)
  • No Ethernet ports because Ethernet is so 2014
  • Right, now that we’ve got the useless D-Link page out of the way, let’s talk about what’s actually in the DAP-1520:

  • MediaTek MT7260A SoC running at 580MHz (includes 2.4GHz radio)
  • 64MB RAM (Winbond W9751G6KB-25 64471X600ZY2)
  • 8MB Flash (MXIC MX 25L640GE)
  • MediaTek MT7610EN (5GHz radio)
  • Skyworks 5GHz Frontend module (datasheet [PDF])
  • This all sounds great, but what do we actually have here? I will preface this post by saying that I started out wanting to port OpenWrt to this device, and I still do, but I got side tracked in my investigation and you’ll have to wait for a follow up post if I ever succeed to port OpenWrt.

    PCB front

    PCB front

    PCB Rear

    PCB Rear

    The UART runs at 57600 8N1.

    No pictures of the power supply because it’s just a boring 5V power source.

    Okay, so now that we know the UART pinout, what does the device say when it boots?

    Boot log:

    U-Boot 1.1.3 (Aug  8 2013 - 10:32:46)
    
    Board: Ralink APSoC DRAM:  64 MB
    relocate_code Pointer at: 83fb0000
    enable ephy clock...done. rf reg 29 = 5
    SSC disabled.
    spi_wait_nsec: 29 
    spi device id: c2 20 17 c2 20 (2017c220)
    find flash: MX25L6405D
    raspi_read: from:30000 len:1000 
    *** Warning - bad CRC, using default environment
    
    ============================================ 
    Ralink UBoot Version: 4.1.1.0
    -------------------------------------------- 
    ASIC 7620_MP (Port5None)
    DRAM component: 512 Mbits DDR, width 16
    DRAM bus: 16 bit
    Total memory: 64 MBytes
    Flash component: SPI Flash
    Date:Aug  8 2013  Time:10:32:46
    Cameo Version: v1.00 Build:01
    Module Name: D-Link DAP-1520A1
    ============================================ 
    icache: sets:512, ways:4, linesz:32 ,total:65536
    dcache: sets:256, ways:4, linesz:32 ,total:32768 
    
     ##### The CPU freq = 580 MHZ #### 
     estimate memory size =64 Mbytes
    raspi_read: from:50000 len:40 
    raspi_read: from:4f0000 len:40 
    
    =================================================
    Check image validation:
    Image1 Header Magic Number --> OK
    Image2 Header Magic Number --> OK
    Image1 Header Checksum --> OK
    Image2 Header Checksum --> OK
    Image1 Data Checksum --> raspi_read: from:50040 len:ddf98 
    OK
    Image2 Data Checksum --> raspi_read: from:4f0040 len:ca8f4 
    OK
    Image1 Stable Flag --> Not stable
    Image1 Try Counter --> 0
    
    Image1: OK Image2: OK
    Both images are OK!!!
    
    =================================================
    
    Please choose the operation: 
       1: Load system code to SDRAM via TFTP. 
       2: Load system code then write to Flash via TFTP. 
       3: Boot system code via Flash (default).
       4: Entr boot command line interface.
       7: Load Boot Loader code then write to Flash via Serial. 
       9: Load Boot Loader code then write to Flash via TFTP. 
     1  0 
       
    3: System Boot system code via Flash.
    ## Booting image at bc050000 ...
    raspi_read: from:50000 len:40 
       Image Name:   Linux Kernel Image
       Image Type:   MIPS Linux Kernel Image (lzma compressed)
       Data Size:    909208 Bytes = 887.9 kB
       Load Address: 80000000
       Entry Point:  8000c310
    raspi_read: from:50040 len:ddf98 
       Verifying Checksum ... OK
       Uncompressing Kernel Image ... OK
    No initrd
    ## Transferring control to Linux (at address 8000c310) ...
    ## Giving linux memsize in MB, 64
    
    Starting kernel ...
    
    
    LINUX started...
    
     THIS IS ASIC
    Linux version 2.6.36.x ([email protected]) (gcc version 3.4.2) #1 Fri Aug 22 16:26:27 CST 2014
    
     The CPU feqenuce set to 580 MHz
    
     MIPS CPU sleep mode enabled.
     PCIE: bypass PCIe DLL.
     PCIE: Elastic buffer control: Addr:0x68 -> 0xB4
     disable all power about PCIe
    CPU revision is: 00019650 (MIPS 24Kc)
    Determined physical RAM map:
     memory: 04000000 @ 00000000 (usable)
    Zone PFN ranges:
      Normal   0x00000000 -> 0x00004000
    Movable zone start PFN for each node
    early_node_map[1] active PFN ranges
        0: 0x00000000 -> 0x00004000
    Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
    Kernel command line: console=ttyS1,57600n8 root=/dev/mtdblock5 console=ttyS0,57600 root=31:05 rootfstype=squashfs init=/sbin/init
    PID hash table entries: 256 (order: -2, 1024 bytes)
    Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
    Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
    Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
    Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
    Writing ErrCtl register=0007efde
    Readback ErrCtl register=0007efde
    Memory: 62028k/65536k available (2225k kernel code, 3508k reserved, 338k data, 168k init, 0k highmem)
    NR_IRQS:128
    MTK/Ralink System Tick Counter init... cd:80271d98, m:214748, s:32
    console [ttyS1] enabled
    Calibrating delay loop... 386.04 BogoMIPS (lpj=772096)
    pid_max: default: 32768 minimum: 301
    Mount-cache hash table entries: 512
    NET: Registered protocol family 16
    RALINK_GPIOMODE = 1a311d
    RALINK_GPIOMODE = 18311d
    PPLL_CFG1=0xe90000
    MT7620 PPLL lock
    PPLL_DRV =0x80080504
    start PCIe register access
    RALINK_PCI_PCICFG_ADDR = 1000f0
    
    *************** MT7620 PCIe RC mode *************
    bio: create slab  at 0
    vgaarb: loaded
    pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x201fffff]
    pci 0000:00:00.0: BAR 1: assigned [mem 0x20200000-0x2020ffff]
    pci 0000:00:00.0: BAR 1: set to [mem 0x20200000-0x2020ffff] (PCI address [0x20200000-0x2020ffff]
    pci 0000:01:00.0: BAR 0: assigned [mem 0x20000000-0x200fffff]
    pci 0000:01:00.0: BAR 0: set to [mem 0x20000000-0x200fffff] (PCI address [0x20000000-0x200fffff]
    pci 0000:01:00.1: BAR 0: assigned [mem 0x20100000-0x201fffff]
    pci 0000:01:00.1: BAR 0: set to [mem 0x20100000-0x201fffff] (PCI address [0x20100000-0x201fffff]
    pci 0000:00:00.0: PCI bridge to [bus 01-01]
    pci 0000:00:00.0:   bridge window [io  disabled]
    pci 0000:00:00.0:   bridge window [mem 0x20000000-0x201fffff]
    pci 0000:00:00.0:   bridge window [mem pref disabled]
    BAR0 at slot 0 = 0
    bus=0x0, slot = 0x0
    res[0]->start = 0
    res[0]->end = 0
    res[1]->start = 20200000
    res[1]->end = 2020ffff
    res[2]->start = 0
    res[2]->end = 0
    res[3]->start = 0
    res[3]->end = 0
    res[4]->start = 0
    res[4]->end = 0
    res[5]->start = 0
    res[5]->end = 0
    bus=0x1, slot = 0x0
    res[0]->start = 20000000
    res[0]->end = 200fffff
    res[1]->start = 0
    res[1]->end = 0
    res[2]->start = 0
    res[2]->end = 0
    res[3]->start = 0
    res[3]->end = 0
    res[4]->start = 0
    res[4]->end = 0
    res[5]->start = 0
    res[5]->end = 0
    bus=0x1, slot = 0x0
    res[0]->start = 20100000
    res[0]->end = 201fffff
    res[1]->start = 0
    res[1]->end = 0
    res[2]->start = 0
    res[2]->end = 0
    res[3]->start = 0
    res[3]->end = 0
    res[4]->start = 0
    res[4]->end = 0
    res[5]->start = 0
    res[5]->end = 0
    Switching to clocksource Ralink external timer
    NET: Registered protocol family 2
    IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
    TCP established hash table entries: 2048 (order: 2, 16384 bytes)
    TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
    TCP: Hash tables configured (established 2048 bind 2048)
    TCP reno registered
    UDP hash table entries: 256 (order: 0, 4096 bytes)
    UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
    NET: Registered protocol family 1
    squashfs: version 4.0 (2009/01/31) Phillip Lougher
    msgmni has been set to 121
    Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
    io scheduler noop registered (default)
    Ralink gpio driver initialized
    Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
    serial8250: ttyS0 at MMIO 0x10000500 (irq = 37) is a 16550A
    serial8250: ttyS1 at MMIO 0x10000c00 (irq = 12) is a 16550A
    brd: module loaded
    deice id : c2 20 17 c2 20 (2017c220)
    MX25L6405D(c2 2017c220) (8192 Kbytes)
    mtd .name = raspi, .size = 0x00800000 (0M) .erasesize = 0x00000008 (0K) .numeraseregions = 65536
    Creating 9 MTD partitions on "raspi":
    0x000000000000-0x000000800000 : "ALL"
    0x000000000000-0x000000030000 : "u-boot"
    0x000000030000-0x000000040000 : "nvram"
    0x000000040000-0x000000050000 : "Factory"
    0x000000050000-0x000000140000 : "linux4"
    0x000000140000-0x0000004e0000 : "rootfs"
    0x0000004e0000-0x0000004f0000 : "LANG"
    0x0000004f0000-0x0000005c0000 : "linux4b"
    0x0000005c0000-0x000000800000 : "rootfsb"
    rdm_major = 253
    SMACCR1 -- : 0x0000000c
    SMACCR0 -- : 0x43762077
    Ralink APSoC Ethernet Driver Initilization. v3.0  256 rx/tx descriptors allocated, mtu = 1500!
    SMACCR1 -- : 0x0000000c
    SMACCR0 -- : 0x43762077
    PROC INIT OK!
    TCP cubic registered
    NET: Registered protocol family 10
    IPv6 over IPv4 tunneling driver
    NET: Registered protocol family 17
    VFS: Mounted root (squashfs filesystem) readonly on device 31:5.
    Freeing unused kernel memory: 168k freed
    init started:  BusyBox v1.01 (2014.08.22-08:26+0000) multi-call binary
    Algorithmics/MIPS FPU Emulator v1.5
    devpts: called with bogus options
    init NVRAM_SPACE from mtdblock size
    init nvram memory map size: 0x10000 order of pages: 0x4
    nvram module init:
        /dev/nvram major number 225 glues to mtd: "nvram" size: 0x00010000
        nvram_space: 0x00010000 mapped via mmap(2)
    openfile :/etc/sysinfo
    openfile :/etc/nvram.default
    
    
    BusyBox v1.01 (2014.08.22-08:26+0000) Built-in shell (ash)
    Enter 'help' for a list of built-in commands.
    
    / # rm: cannot remove `/var/wizard_lang.js': No such file or directory
    umount: cannot umount /tmp/lang_pack: No such file or directory
    eth2: Cannot assign requested address
    Raeth v3.0 (Tasklet,SkbRecycle)
    
    phy_tx_ring = 0x03f4b000, tx_ring = 0xa3f4b000
    
    phy_rx_ring0 = 0x03f4c000, rx_ring0 = 0xa3f4c000
    SMACCR1 -- : 0x000054b8
    SMACCR0 -- : 0x0a7d19a6
    CDMA_CSG_CFG = 81000000
    GDMA1_FWD_CFG = 20710000
    umount: cannot umount /tmp/lang_pack: No such file or directory
    mount: mounting /dev/mtdblock6 on /tmp/lang_pack failed
    eth2: Cannot assign requested address
    device eth2 entered promiscuous mode
    TFTP main
    standard_tftp_server launched on port 69.
    killall: syslogd: no process killed
    killall: klogd: no process killed
    Sat Jan  1 00:00:00 UTC 2011
    /tmp/password has been created
    br0: port 1(eth2) entering forwarding state
    br0: port 1(eth2) entering forwarding state
    Set: phy[0].reg[0] = 3900
    Set: phy[1].reg[0] = 3900
    Set: phy[2].reg[0] = 3900
    Set: phy[3].reg[0] = 3900
    Set: phy[4].reg[0] = 3900
    Set: phy[0].reg[0] = 3100
    2011-01-01 00:00:00: (network.c.247) warning: please use server.use-ipv6 only for hostnames, not without server.bind / empty address; your config will break if the kernel default for IPV6_V6ONLY changes 
    rt2860v2_ap: module license 'unspecified' taints kernel.
    Disabling lock debugging due to kernel taint
    
    
    === pAd = c04cd000, size = 1278080 ===
    
    <-- RTMPAllocTxRxRingMemory, Status=0
    <-- RTMPAllocAdapterBlock, Status=0
    AP Driver version-2.7.1.6_edcca_monitor_20131222
    
    
    === pAd = c0b02000, size = 2010752 ===
    
    <-- RTMPAllocTxRxRingMemory, Status=0
    MT76x0_WLAN_ChipOnOff(): OnOff:1, pAd->WlanFunCtrl:0x0, Reg-WlanFunCtrl=0xff000002
    MACVersion = 0x76502000
    RX DESC a3672000  size = 2048
    RTMP_TimerListAdd: add timer obj c05a1e20!
    RTMP_TimerListAdd: add timer obj c053b694!
    RTMP_TimerListAdd: add timer obj c053f78c!
    RTMP_TimerListAdd: add timer obj c053f84c!
    RTMP_TimerListAdd: add timer obj c053f90c!
    RTMP_TimerListAdd: add timer obj c053f9cc!
    RTMP_TimerListAdd: add timer obj c053fa8c!
    RTMP_TimerListAdd: add timer obj c053fb4c!
    RTMP_TimerListAdd: add timer obj c053fc0c!
    RTMP_TimerListAdd: add timer obj c053fccc!
    RTMP_TimerListAdd: add timer obj c053fd8c!
    RTMP_TimerListAdd: add timer obj c053fe4c!
    RTMP_TimerListAdd: add timer obj c053ff0c!
    RTMP_TimerListAdd: add timer obj c053ffcc!
    RTMP_TimerListAdd: add timer obj c054008c!
    RTMP_TimerListAdd: add timer obj c054014c!
    RTMP_TimerListAdd: add timer obj c054020c!
    RTMP_TimerListAdd: add timer obj c05402cc!
    RTMP_TimerListAdd: add timer obj c0569e9c!
    RTMP_TimerListAdd: add timer obj c056df94!
    RTMP_TimerListAdd: add timer obj c056e054!
    RTMP_TimerListAdd: add timer obj c056e114!
    RTMP_TimerListAdd: add timer obj c056e1d4!
    RTMP_TimerListAdd: add timer obj c056e294!
    RTMP_TimerListAdd: add timer obj c056e354!
    RTMP_TimerListAdd: add timer obj c056e414!
    RTMP_TimerListAdd: add timer obj c056e4d4!
    RTMP_TimerListAdd: add timer obj c056e594!
    RTMP_TimerListAdd: add timer obj c056e654!
    RTMP_TimerListAdd: add timer obj c056e714!
    RTMP_TimerListAdd: add timer obj c056e7d4!
    RTMP_TimerListAdd: add timer obj c056e894!
    RTMP_TimerListAdd: add timer obj c056e954!
    RTMP_TimerListAdd: add timer obj c056ea14!
    RTMP_TimerListAdd: add timer obj c056ead4!
    RTMP_TimerListAdd: add timer obj c053b668!
    RTMP_TimerListAdd: add timer obj c053b6c0!
    RTMP_TimerListAdd: add timer obj c053f760!
    RTMP_TimerListAdd: add timer obj c053f820!
    RTMP_TimerListAdd: add timer obj c053f8e0!
    RTMP_TimerListAdd: add timer obj c053f9a0!
    RTMP_TimerListAdd: add timer obj c053fa60!
    RTMP_TimerListAdd: add timer obj c053fb20!
    RTMP_TimerListAdd: add timer obj c053fbe0!
    RTMP_TimerListAdd: add timer obj c053fca0!
    RTMP_TimerListAdd: add timer obj c053fd60!
    RTMP_TimerListAdd: add timer obj c053fe20!
    RTMP_TimerListAdd: add timer obj c053fee0!
    RTMP_TimerListAdd: add timer obj c053ffa0!
    RTMP_TimerListAdd: add timer obj c0540060!
    RTMP_TimerListAdd: add timer obj c0540120!
    RTMP_TimerListAdd: add timer obj c05401e0!
    RTMP_TimerListAdd: add timer obj c05402a0!
    RTMP_TimerListAdd: add timer obj c0569e70!
    RTMP_TimerListAdd: add timer obj c0569ec8!
    RTMP_TimerListAdd: add timer obj c056df68!
    RTMP_TimerListAdd: add timer obj c056e028!
    RTMP_TimerListAdd: add timer obj c056e0e8!
    RTMP_TimerListAdd: add timer obj c056e1a8!
    RTMP_TimerListAdd: add timer obj c056e268!
    RTMP_TimerListAdd: add timer obj c056e328!
    RTMP_TimerListAdd: add timer obj c056e3e8!
    RTMP_TimerListAdd: add timer obj c056e4a8!
    RTMP_TimerListAdd: add timer obj c056e568!
    RTMP_TimerListAdd: add timer obj c056e628!
    RTMP_TimerListAdd: add timer obj c056e6e8!
    RTMP_TimerListAdd: add timer obj c056e7a8!
    RTMP_TimerListAdd: add timer obj c056e868!
    RTMP_TimerListAdd: add timer obj c056e928!
    RTMP_TimerListAdd: add timer obj c056e9e8!
    RTMP_TimerListAdd: add timer obj c056eaa8!
    RTMP_TimerListAdd: add timer obj c053b63c!
    RTMP_TimerListAdd: add timer obj c0569e44!
    RTMP_TimerListAdd: add timer obj c053f7b8!
    RTMP_TimerListAdd: add timer obj c053f878!
    RTMP_TimerListAdd: add timer obj c053f938!
    RTMP_TimerListAdd: add timer obj c053f9f8!
    RTMP_TimerListAdd: add timer obj c053fab8!
    RTMP_TimerListAdd: add timer obj c053fb78!
    RTMP_TimerListAdd: add timer obj c053fc38!
    RTMP_TimerListAdd: add timer obj c053fcf8!
    RTMP_TimerListAdd: add timer obj c053fdb8!
    RTMP_TimerListAdd: add timer obj c053fe78!
    RTMP_TimerListAdd: add timer obj c053ff38!
    RTMP_TimerListAdd: add timer obj c053fff8!
    RTMP_TimerListAdd: add timer obj c05400b8!
    RTMP_TimerListAdd: add timer obj c0540178!
    RTMP_TimerListAdd: add timer obj c0540238!
    RTMP_TimerListAdd: add timer obj c05402f8!
    RTMP_TimerListAdd: add timer obj c053b710!
    RTMP_TimerListAdd: add timer obj c053b73c!
    RTMP_TimerListAdd: add timer obj c053b768!
    RTMP_TimerListAdd: add timer obj c056dfc0!
    RTMP_TimerListAdd: add timer obj c056e080!
    RTMP_TimerListAdd: add timer obj c056e140!
    RTMP_TimerListAdd: add timer obj c056e200!
    RTMP_TimerListAdd: add timer obj c056e2c0!
    RTMP_TimerListAdd: add timer obj c056e380!
    RTMP_TimerListAdd: add timer obj c056e440!
    RTMP_TimerListAdd: add timer obj c056e500!
    RTMP_TimerListAdd: add timer obj c056e5c0!
    RTMP_TimerListAdd: add timer obj c056e680!
    RTMP_TimerListAdd: add timer obj c056e740!
    RTMP_TimerListAdd: add timer obj c056e800!
    RTMP_TimerListAdd: add timer obj c056e8c0!
    RTMP_TimerListAdd: add timer obj c056e980!
    RTMP_TimerListAdd: add timer obj c056ea40!
    RTMP_TimerListAdd: add timer obj c056eb00!
    RTMP_TimerListAdd: add timer obj c0569f18!
    RTMP_TimerListAdd: add timer obj c0569f44!
    RTMP_TimerListAdd: add timer obj c0569f70!
    RTMP_TimerListAdd: add timer obj c04d5014!
    RTMP_TimerListAdd: add timer obj c04d4bf8!
    RTMP_TimerListAdd: add timer obj c04d4fe4!
    RTMP_TimerListAdd: add timer obj c04d5320!
    RTMP_TimerListAdd: add timer obj c04d5260!
    RTMP_TimerListAdd: add timer obj c04d5290!
    RTMP_TimerListAdd: add timer obj c04d8fbc!
    RTMP_TimerListAdd: add timer obj c04d8ba0!
    RTMP_TimerListAdd: add timer obj c04d8f8c!
    RTMP_TimerListAdd: add timer obj c04d92c8!
    RTMP_TimerListAdd: add timer obj c04d9208!
    RTMP_TimerListAdd: add timer obj c04d9238!
    RTMP_TimerListAdd: add timer obj c04dcf64!
    RTMP_TimerListAdd: add timer obj c04dcb48!
    RTMP_TimerListAdd: add timer obj c04dcf34!
    RTMP_TimerListAdd: add timer obj c04dd270!
    RTMP_TimerListAdd: add timer obj c04dd1b0!
    RTMP_TimerListAdd: add timer obj c04dd1e0!
    RTMP_TimerListAdd: add timer obj c04e0f0c!
    RTMP_TimerListAdd: add timer obj c04e0af0!
    RTMP_TimerListAdd: add timer obj c04e0edc!
    RTMP_TimerListAdd: add timer obj c04e1218!
    RTMP_TimerListAdd: add timer obj c04e1158!
    RTMP_TimerListAdd: add timer obj c04e1188!
    RTMP_TimerListAdd: add timer obj c04e4eb4!
    RTMP_TimerListAdd: add timer obj c04e4a98!
    RTMP_TimerListAdd: add timer obj c04e4e84!
    RTMP_TimerListAdd: add timer obj c04e51c0!
    RTMP_TimerListAdd: add timer obj c04e5100!
    RTMP_TimerListAdd: add timer obj c04e5130!
    RTMP_TimerListAdd: add timer obj c04e8e5c!
    RTMP_TimerListAdd: add timer obj c04e8a40!
    RTMP_TimerListAdd: add timer obj c04e8e2c!
    RTMP_TimerListAdd: add timer obj c04e9168!
    RTMP_TimerListAdd: add timer obj c04e90a8!
    RTMP_TimerListAdd: add timer obj c04e90d8!
    RTMP_TimerListAdd: add timer obj c04ece04!
    RTMP_TimerListAdd: add timer obj c04ec9e8!
    RTMP_TimerListAdd: add timer obj c04ecdd4!
    RTMP_TimerListAdd: add timer obj c04ed110!
    RTMP_TimerListAdd: add timer obj c04ed050!
    RTMP_TimerListAdd: add timer obj c04ed080!
    RTMP_TimerListAdd: add timer obj c04f0dac!
    RTMP_TimerListAdd: add timer obj c04f0990!
    RTMP_TimerListAdd: add timer obj c04f0d7c!
    RTMP_TimerListAdd: add timer obj c04f10b8!
    RTMP_TimerListAdd: add timer obj c04f0ff8!
    RTMP_TimerListAdd: add timer obj c04f1028!
    RTMP_TimerListAdd: add timer obj c053e5ec!
    RTMP_TimerListAdd: add timer obj c053e1d0!
    RTMP_TimerListAdd: add timer obj c053e5bc!
    RTMP_TimerListAdd: add timer obj c053e8f8!
    RTMP_TimerListAdd: add timer obj c053e61c!
    RTMP_TimerListAdd: add timer obj c053e64c!
    RTMP_TimerListAdd: add timer obj c053e67c!
    RTMP_TimerListAdd: add timer obj c056cdf4!
    RTMP_TimerListAdd: add timer obj c056c9d8!
    RTMP_TimerListAdd: add timer obj c056cdc4!
    RTMP_TimerListAdd: add timer obj c056d100!
    RTMP_TimerListAdd: add timer obj c056ce24!
    RTMP_TimerListAdd: add timer obj c056ce54!
    RTMP_TimerListAdd: add timer obj c056ce84!
    RTMP_TimerListAdd: add timer obj c057878c!
    RTMP_TimerListAdd: add timer obj c05788a8!
    RTMP_TimerListAdd: add timer obj c05787b8!
    RTMP_TimerListAdd: add timer obj c056fcc4!
    RTMP_TimerListAdd: add timer obj c04d24c4!
    RTMP_TimerListAdd: add timer obj c04d646c!
    RTMP_TimerListAdd: add timer obj c04da414!
    RTMP_TimerListAdd: add timer obj c04de3bc!
    RTMP_TimerListAdd: add timer obj c04e2364!
    RTMP_TimerListAdd: add timer obj c04e630c!
    RTMP_TimerListAdd: add timer obj c04ea2b4!
    RTMP_TimerListAdd: add timer obj c04ee25c!
    RTMP_TimerListAdd: add timer obj c056f9d0!
    APSDCapable[0]=0
    APSDCapable[1]=0
    APSDCapable[2]=0
    APSDCapable[3]=0
    APSDCapable[4]=0
    APSDCapable[5]=0
    APSDCapable[6]=0
    APSDCapable[7]=0
    APSDCapable[8]=0
    APSDCapable[9]=0
    APSDCapable[10]=0
    APSDCapable[11]=0
    APSDCapable[12]=0
    APSDCapable[13]=0
    APSDCapable[14]=0
    APSDCapable[15]=0
    default ApCliAPSDCapable[0]=0
    default ApCliAPSDCapable[1]=0
    start ch = 1, ch->num = 2
    30 30 30 30 
    30 30 30 30 30 30 30 30 
    26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
    start ch = 3, ch->num = 9
    30 30 30 30 
    30 30 30 30 30 30 30 30 
    26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 
    26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 
    start ch = 12, ch->num = 2
    30 30 30 30 
    30 30 30 30 30 30 30 30 
    26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
    start ch = 14, ch->num = 1
    30 30 30 30 
    0 0 0 0 0 0 0 0 
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
    0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
    1. Phy Mode = 9
    2. Phy Mode = 9
    E2PROM: D0 target power=0xff20 
    E2PROM: 40 MW Power Delta= 0 
    3. Phy Mode = 9
    AntCfgInit: primary/secondary ant 0/1
    Initialize RF Central Registers for E2 !!!
    Initialize RF Central Registers for E3 !!!
    Initialize RF Channel Registers for E2 !!!
    Initialize RF Channel Registers for E3 !!!
    Initialize RF DCCal Registers for E2 !!!
    Initialize RF DCCal Registers for E3 !!!
    D1 = -1, D2 = 16, CalCode = 40 !!!
    RT6352_Temperature_Init : BBPR49 = 0xffffffff
    RT6352_Temperature_Init : TemperatureRef25C = 0xfffffff5
    Current Temperature from BBP_R49=0xffffffec
    RT6352_TemperatureCalibration:: CurrentTemper 
    @@@ ed_monitor_init : <===
    Main bssid = 54:b8:0a:7d:19:a6
    
    @@@ ed_monitor_init : num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 38, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 40, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 42, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 44, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 46, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 48, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 52, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 54, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 56, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 58, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 60, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 62, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 64, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 100, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 102, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 104, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 106, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 108, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 110, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 112, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 116, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 118, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 120, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 122, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 124, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 126, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 128, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 132, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 134, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 136, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 140, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 149, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 151, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 153, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 155, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 157, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 159, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 161, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 165, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 169, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    start ch = 173, ch->num = 1
    0 0 0 0 
    38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    38 38 38 38 36 36 32 32 38 38 38 38 36 36 32 32 
    32 32 32 32 32 32 32 32 26 26 
    1. Phy Mode = 49
    2. Phy Mode = 49
    ext_pa_current_setting = 1
    3. Phy Mode = 49
    AntCfgInit: primary/secondary ant 0/1
    ChipStructAssign(): RALINK6590 hook !
    MCS Set = ff 00 00 00 01
    MT76x0_ChipBBPAdjust():rf_bw=2, ext_ch=1, PrimCh=36, HT-CentCh=38, VHT-CentCh=42
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    APStartUp(): AP Set CentralFreq at 42(Prim=36, HT-CentCh=38, VHT-CentCh=42, BBP_BW=2)
    Main bssid = 54:b8:0a:7d:19:a8
    <==== rt28xx_init, Status=0
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    RT6352_TemperatureCalibration:: CurrentTemper < 20 
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    MT76x0_ChipBBPAdjust():rf_bw=2, ext_ch=1, PrimCh=44, HT-CentCh=46, VHT-CentCh=42
    MT76x0_ChipSwitchChannel: DefaultTargetPwr = 30
    APStartUp(): AP Set CentralFreq at 42(Prim=44, HT-CentCh=46, VHT-CentCh=42, BBP_BW=2)
    0x1300 = 00064380
    RTMPDrvOpen(1):Check if PDMA is idle!
    RTMPDrvOpen(2):Check if PDMA is idle!
    device rai0 entered promiscuous mode
    br0: port 4(rai0) entering forwarding state
    br0: port 4(rai0) entering forwarding state
    device apclii0 entered promiscuous mode
    br0: port 5(apclii0) entering forwarding state
    br0: port 5(apclii0) entering forwarding state
    Interface doesn't accept private ioctl...
    set (8BE2): Invalid argument
    killall: udhcpc: no process killed
    SIOCSIFFLAGS: Cannot assign requested address
    rm: cannot remove `/var/tmp/previous_dn': No such file or directory
    rm: cannot remove `/var/tmp/previous_dns': No such file or directory
    rm: cannot remove `/var/tmp/m_flag': No such file or directory
    rm: cannot remove `/var/tmp/o_flag': No such file or directory
    RTNETLINK answers: No such file or directory
    cat: /var/etc/resolv.conf: No such file or directory
    sh: cannot create /proc/sys/net/ipv6/conf/br0/disable_ipv6: Directory nonexistent
    Start IPv6 dhclient
    Sat Jan  1 00:00:00 UTC 2011
    rdnssd is already active !
    RT6352_TemperatureCalibration:: CurrentTemper < 20 
    Start IPv6 dhclient
    DHCP server start.
    device_lan_ip=192.168.0.50 , device_lan_subnet_mask=255.255.255.0
    max_leases value (254) not sane, setting to 20 instead
    Unable to open /var/misc/udhcpd.leases for reading
    llmnr: have no available linklocal address. wait count=0
    /tmp/password has been created
    2011-01-01 00:00:00: (network.c.247) warning: please use server.use-ipv6 only for hostnames, not without server.bind / empty address; your config will break if the kernel default for IPV6_V6ONLY changes 
    Failed to kill daemon: No such file or directory
    Daemon already running on PID 315
    RT6352_TemperatureCalibration:: CurrentTemper < 20 
    RT6352_TemperatureCalibration:: CurrentTemper < 20 
    RT6352_TemperatureCalibration:: CurrentTemper < 20
    

    Warning: Do not attempt to modify the firmware of this device if you do not have hardware to rewrite the firmware to the SPI flash. Before I even powered up the device the first time, I took a dump of the SPI flash in case I ended up “bricking” the device (which I did, many times).

    You can build your own SPI flash reader/writer with a Teensy and a chip clip. I am using the work of Trammell Hudson who gave an awesome talk at 31C3 on manipulating UEFI on MacBook Pros for fun and profit.

    You can find a copy of the SPI dump of my device (firmware 1.05) here. You cannot flash this image without hardware tools as described above. If you flash this dump, your device will have the same MAC address as mine. This dump should be used only as an option of last resort.

    Poking around the D-Link firmware for vulnerabilities

    I would love to say that I’m an infosec god, and that I can hack anything that moves. Really though, I’m not. I tried to find exploits for D-Link, and it doesn’t seem that there is any shortage of HNAP exploits and other nasty things, but I was unable to get the device to do any interesting things for me, like start a telnet server.

    Disassembling the firmware to learn more about installed software

    Since my Google-fu is weak, I couldn’t find the firmware images for this device on D-Link’s website at first, so I just disassembled the firmware I dumped from the MXIC.

    $ binwalk dlink-dap1520.bin 
    
    DECIMAL       HEXADECIMAL     DESCRIPTION
    --------------------------------------------------------------------------------
    99968         0x18680         U-Boot version string, "U-Boot 1.1.3 (Aug  8 2013 - 10:32:46)"
    100732        0x1897C         HTML document header
    101832        0x18DC8         HTML document footer
    101954        0x18E42         HTML document header
    102754        0x19162         HTML document footer
    102878        0x191DE         HTML document header
    105248        0x19B20         HTML document footer
    105367        0x19B97         HTML document header
    106050        0x19E42         HTML document footer
    106174        0x19EBE         HTML document header
    106255        0x19F0F         HTML document footer
    196962        0x30162         Unix path: /01/01/00/00/00
    327680        0x50000         uImage header, header size: 64 bytes, header CRC: 0xC9616E23, created: 2014-08-22 08:41:24, image size: 909208 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0x895D3AE, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
    327744        0x50040         LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 2798288 bytes
    1310720       0x140000        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 3459080 bytes, 649 inodes, blocksize: 65536 bytes, created: 2014-08-22 08:41:35
    5177344       0x4F0000        uImage header, header size: 64 bytes, header CRC: 0x225D8E97, created: 2013-09-26 08:58:51, image size: 829684 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0xA98529B2, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
    5177408       0x4F0040        LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 2544052 bytes
    6029312       0x5C0000        Squashfs filesystem, little endian, version 4.0, compression:xz, size: 2192260 bytes, 345 inodes, blocksize: 65536 bytes, created: 2013-09-26 08:59:04
    

    Something interesting, there are two Squashfs filesystems on this device. This makes some sense, given what we saw earlier in the uboot logs:

    Check image validation:
    Image1 Header Magic Number --> OK
    Image2 Header Magic Number --> OK
    Image1 Header Checksum --> OK
    Image2 Header Checksum --> OK
    Image1 Data Checksum --> raspi_read: from:50040 len:ddf98 
    OK
    Image2 Data Checksum --> raspi_read: from:4f0040 len:ca8f4 
    OK
    Image1 Stable Flag --> Not stable
    Image1 Try Counter --> 0
    
    Image1: OK Image2: OK
    Both images are OK!!!

    Using dd, we can extract both Squashfs images from the firmware file. I used my dump, but actually I would recommend you just head over to D-Link's website and download the 1.06 firmware image [ZIP] and dump that instead. However, D-Link's firmware is missing the second Squashfs filesystem.

    Squashfs #1

    $ dd if=dlink-dap1520.bin of=squashfs1.bin bs=1 skip=1310720

    Squashfs #2

    $ dd if=dlink-dap1520.bin of=squashfs2.bin bs=1 skip=6029312

    Run the 'ol unsquashfs on squashfs1.bin and squashfs2.bin, and you'll have the extracted filesystems of the squashfs images in my dump the firmware. Remember to rename the directory squashfs-root between runs, or specify unsquashfs -d with a different directory name to decompress the images into respective directories.

    If you're using the D-Link firmware from their website, the dd command is a bit different due to offsets and all:

    $ binwalk DAP1520A1_FW106B04.bin
    
    DECIMAL       HEXADECIMAL     DESCRIPTION
    --------------------------------------------------------------------------------
    0             0x0             uImage header, header size: 64 bytes, header CRC: 0xBA3B64BA, created: 2015-01-22 03:48:48, image size: 909200 bytes, Data Address: 0x80000000, Entry Point: 0x8000C310, data CRC: 0x310BA125, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
    64            0x40            LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 2798288 bytes
    983040        0xF0000         Squashfs filesystem, little endian, version 4.0, compression:xz, size: 3460300 bytes, 649 inodes, blocksize: 65536 bytes, created: 2015-01-22 03:48:54
    $ dd if=DAP1520A1_FW106B04.bin of=dlink106.bin bs=1 skip=983040

    Now unsquashfs that, and you'll have firmware 1.06 from D-Link.

    I'm going to leave investigation of the individual files in the firmware to the reader, but I'd like to state some facts I learned while investigating the firmware:

  • There are two copies of busybox on the firmware, versions 1.01 and 1.6.1. Some programs in /bin are linked to 1.01 and others to 1.6.1. I have no idea why D-Link would do this.
  • Pretty much everything on the device is run from /bin/cli and /bin/ssi. Other people on the web have analyzed these binaries and can tell you what they do (and how insecure they are).
  • The second squashfs image is the D-Link recovery OS. This OS will boot if the first kernel fails the integrity check performed in uboot. Hilariously, it won't boot into the recovery environment if you flash a bad kernel to the device in Image 1 as I found out.

    You might have noticed the flash layout from the binwalk of the firmware dump I made, but here is the actual firmware layout as reported by Linux:

    0x000000000000-0x000000800000 : "ALL"
    0x000000000000-0x000000030000 : "u-boot"
    0x000000030000-0x000000040000 : "nvram"
    0x000000040000-0x000000050000 : "Factory"
    0x000000050000-0x000000140000 : "linux4"
    0x000000140000-0x0000004e0000 : "rootfs"
    0x0000004e0000-0x0000004f0000 : "LANG"
    0x0000004f0000-0x0000005c0000 : "linux4b"
    0x0000005c0000-0x000000800000 : "rootfsb"

    To summarize:
    ALL: This spans from 0x000000 to 0x800000 which is the entire 8MB of the MXIC chip
    u-boot: u-boot loader
    nvram: Storage space for configuration variables. More on this in part 2
    Factory: No idea.
    linux4: This is the primary kernel on the device, and the one that will boot if your device has a valid Image 1. This is the firmware that you download from D-Link's website. Despite the label, it is not Linux 4.x, but 2.6.36.
    rootfs: Squashfs compressed filesystem of the primary OS (Image 1)
    LANG: No idea.
    linux4b: Recovery kernel. This kernel will be booted if Image 1 kernel fails verification.
    rootfsb: Squashfs compressed recovery filesystem. This, along with linux4b boot if Image 1 is corrupt and allow you to flash a firmware through the web interface to restore the device.

    I must say that the inclusion of a recovery OS is an interesting move on D-Link's part. Since I don't buy their products normally, I'm not sure if other D-Link devices also have this recovery OS on them. It seems like a good idea to include on this device, since if a firmware update fails, since there are no Ethernet ports on the device it's not possible to recover via TFTP, as it would be on a normal router. The firmware update from D-Link's website only updates Image 1 squashfs and kernel. Image 2 on my device is firmware version 1.00, and the squashfs filesystem is smaller than the Image 1 OS.

    If you do some maths on the mtd blocks, you will see that with the stock D-Link layout, the Image 1 kernel can only be 983040 bytes (0xF0000) large. Any larger, and the kernel will not fit in flash. The recovery kernel has to be even smaller, maximum 851968 bytes (0xD0000).

    Since this device lacks Ethernet ports, it doesn't include some of the features one would consider necessary on a home router, such as port forwarding, firewall configuration, and the like. I suspect that not needing to include these features gave D-Link the space on flash to store a recovery OS. As you can see though, they did have to make some compromises in the allocation of flash to fit the main and recovery OS within 8MB. The device does not function as a WiFi repeater in the recovery OS, only allowing you to reflash a firmware.

    As much as I would love to cram all of what I did into one post, this is getting long already.

    Stay tuned for part 2 where I compile the D-Link GPL firmware from source and backdoor the device to allow shell access without a login (infosec is hard). If you've heard horror stories about GPL firmwares before, they're all true...

    Come back soon!

    Arch Linux and SDIO WiFi on a Bay Trail tablet

    tl;dr If you just came to download the bootable USB stick filesystem to boot your tablet, click here.

    You will need to format a USB key (minimum 1GB) with a VFAT/FAT32 filesystem with the label ARCH_201512, unzip the contents of the file to the USB key, and read the section marked Grub near the bottom of this post to boot! It shouldn’t require Linux to set up the USB key.

    I highly recommend you make a backup of the tablet before you proceed to install Linux. The easiest/fastest/laziest way I have found is to use dd and pigz to make a block for block backup of the internal EMMC onto an ext4 formatted microSD card (as the archive will exceed the 4GB limit of VFAT).


    So, you have a Bay Trail based tablet, in my case a Dell Venue 8 Pro (model 3845), and you want to install Linux on it. Chances are pretty good that your tablet will use SDIO for WiFi, and this means that you will start the installer and quickly realize you have no WiFi. Bummer. Hope you’ve got a USB to Ethernet adapter with you, and a USB OTG hub with 3 ports.

    Or, you could compile a custom kernel with patches for the SDIO WiFi chipset, put it into the Arch Linux installer, and then have glorious WiFi for your installation.

    I chose the second option, because USB ethernet adapters are slow. And now I will tell you how I did it, so you too can do it too.

    First: you need to have a computer which can build a normal Linux kernel. I run Arch Linux also on my laptop, so just install the development tools and you can start:

    $ sudo pacman -S base-devel arch-install-scripts squashfs-tools
    

    Go download the latest stable Linux kernel from kernel.org, I used the following: https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.3.2.tar.xz

    Then you need to download the source code for the rtl8723bs WiFi chipset module (it is not in mainline yet):
    https://github.com/hadess/rtl8723bs

    Decompress the Linux source you downloaded earlier:

    $ tar -xf linux-4.3.2.tar.xz
    

    And decompress the rtl8723bs driver you downloaded earlier:

    $ unzip rtl8723bs-master.zip
    

    Don’t forget to apply the patches from the rtl8723bs driver:

    $ cd linux-4.3.2
    linux-4.3.2 ~$ patch -p1 < ../rtl8723bs-master/patches/0001-PM-QoS-Add-pm_qos_cancel_request_lazy-that-doesn-t-s.patch 
    patching file include/linux/pm_qos.h
    patching file kernel/power/qos.c
    linux-4.3.2 ~$ patch -p1 < ../rtl8723bs-master/patches/0001-mmc-sdhci-get-runtime-pm-when-sdio-irq-is-enabled.patch    
    patching file drivers/mmc/host/sdhci.c
    Hunk #1 succeeded at 1731 (offset -13 lines).
    Hunk #2 succeeded at 1743 (offset -13 lines).
    linux-4.3.2 ~$ patch -p1 < ../rtl8723bs-master/patches/0002-mmc-sdhci-Support-maximum-DMA-latency-request-via-PM.patch 
    patching file drivers/mmc/host/sdhci.c
    Hunk #2 succeeded at 1402 (offset 2 lines).
    Hunk #3 succeeded at 1427 (offset 2 lines).
    Hunk #4 succeeded at 2206 (offset 2 lines).
    Hunk #5 succeeded at 2279 (offset 2 lines).
    Hunk #6 succeeded at 2911 (offset 2 lines).
    Hunk #7 succeeded at 3407 (offset 2 lines).
    Hunk #8 succeeded at 3472 (offset 2 lines).
    Hunk #9 succeeded at 3529 (offset 2 lines).
    patching file drivers/mmc/host/sdhci.h
    Hunk #2 succeeded at 428 (offset 5 lines).
    linux-4.3.2 ~$ patch -p1 < ../rtl8723bs-master/patches/0003-mmc-sdhci-acpi-Fix-device-hang-on-Intel-BayTrail.patch     
    patching file drivers/mmc/host/sdhci-acpi.c
    linux-4.3.2 ~$ patch -p1 < ../rtl8723bs-master/patches/0004-mmc-sdhci-pci-Fix-device-hang-on-Intel-BayTrail.patch  
    patching file drivers/mmc/host/sdhci-pci.c
    

    If any of the patches fail to apply, do not proceed with building the kernel, you will not build a working kernel with SDIO WiFi support.

    Moving right along, I stole the stock Arch Linux configuration from the 2015.12 installer ISO and ran make oldconfig to bring it up to date on Linux 4.3.2.

    Here is a copy of the .config which you will want to use. The .config is inside the zip file, just move the zip file to the linux-4.3.2 directory and unzip.

    Verify that everything is cool with the .config file you decompressed (if you use a newer kernel this will prompt you to answer questions about new features supported which are not in the config file):

    linux-4.3.2 ~$ make oldconfig
    scripts/kconfig/conf  --oldconfig Kconfig
    #
    # configuration written to .config
    #
    linux-4.3.2 ~$ make -j 9
    

    Now wait a really long time. I will never understand why Arch Linux includes kernel modules for USB webcams in their text-only installer media…

    Now, while this is happening, download the latest Arch Linux live installation media, because we’re going to open it up and replace the kernel and squashfs:
    https://www.archlinux.org/download/

    I followed the excellent Arch Wiki instructions to remaster the install ISO:
    https://wiki.archlinux.org/index.php/Remastering_the_Install_ISO

    Mount the ISO somewhere:

    $ mkdir /tmp/archlinux-iso
    $ sudo mount -o loop archlinux-2015.12.01-dual.iso /tmp/archlinux-iso
    

    Since I have 16GB of RAM, I just do everything in /tmp because it’s a ramdisk and faster than an SSD:

    $ cp /tmp/archlinux-iso/arch/x86_64/airootfs.sfs /tmp/
    $ cd /tmp/
    $ unsquashfs airootfs.sfs
    

    Now, hopefully by now your kernel has finished building and we can install it to the recently unsquashed install ISO:

    linux-4.3.2 ~$ sudo make INSTALL_MOD_PATH=/tmp/squashfs-root modules_install
    linux-4.3.2 ~$ sudo cp arch/x86/boot/bzImage /tmp/squashfs-root/boot/vmlinuz
    

    This will install our kernel modules to the squashfs-root folder. Feel free to delete the modules from the previous kernel version if you want to save space (for me this was 4.2.5-1-ARCH):

    $ sudo rm -rf /tmp/squashfs-root/lib/modules/4.2.5-1-ARCH/
    

    Now, we need to build the rtl8723bs module:

    $ cd rtl8723bs-master
    rtl8723bs-master ~$ make KSRC=~/linux-4.3.2 KVER=4.3.2-ARCH
      (output omitted for brevity)
      Building modules, stage 2.
      MODPOST 1 modules
      CC      /home/hmartin/rtl8723bs-master/r8723bs.mod.o
      LD [M]  /home/hmartin/rtl8723bs-master/r8723bs.ko
    make[1]: Leaving directory '/home/hmartin/linux-4.3.2'
    rtl8723bs-master ~$ sudo cp r8723bs.ko /tmp/squashfs-root/lib/modules/4.3.2-ARCH/kernel/drivers/net/wireless/
    rtl8723bs-master ~$ sudo chmod 0644 /tmp/squashfs-root/lib/modules/4.3.2-ARCH/kernel/drivers/net/wireless/r8723bs.ko
    rtl8723bs-master ~$ sudo cp -n rtl8723bs_nic.bin /tmp/squashfs-root/lib/firmware/rtlwifi/rtl8723bs_nic.bin
    rtl8723bs-master ~$ sudo cp -n rtl8723bs_wowlan.bin /tmp/squashfs-root/lib/firmware/rtlwifi/rtl8723bs_wowlan.bin
    

    Okay, now we need to chroot into the decompressed squashfs filesystem to create an initrd. We need to modify /etc/mkinitcpio.conf in the squashfs root so we can generate an initrd with the correct modules and options, otherwise your tablet won’t boot with the new kernel:

    $ sudo arch-chroot /tmp/squashfs-root
    (chroot) $ depmod -a 4.3.2-ARCH
    (chroot) $ vi /etc/mkinitcpio.conf
    - MODULES=""
    + MODULES="r8723bs"
    - HOOKS="base udev autodetect modconf block filesystems keyboard fsck"
    + HOOKS="base udev memdisk archiso_shutdown archiso archiso_loop_mnt archiso_pxe_common archiso_pxe_nbd archiso_pxe_http archiso_pxe_nfs archiso_k
    ms block pcmcia filesystems keyboard"
    - #COMPRESSION="xz"
    + COMPRESSION="xz"
    

    Earlier we installed the 4.3.2-ARCH kernel modules, and also copied the kernel to /boot/ within the decompressed squashfs filesystem. Now we are going to use the modules, the vmlinuz kernel in /tmp/squashfs-root/boot/, and the above modifications to the /etc/mkinitcpio.conf file to generate a new initrd which we will call archiso.img:

    (chroot) $ mkinitcpio -k /boot/vmlinuz -c /etc/mkinitcpio.conf -g /boot/archiso.img -k 4.3.2-ARCH
    ==> Starting build: 4.3.2-ARCH
      -> Running build hook: [base]
      -> Running build hook: [udev]
      -> Running build hook: [memdisk]
      -> Running build hook: [archiso_shutdown]
      -> Running build hook: [archiso]
      -> Running build hook: [archiso_loop_mnt]
      -> Running build hook: [archiso_pxe_common]
    ==> WARNING: Possibly missing firmware for module: liquidio
      -> Running build hook: [archiso_pxe_nbd]
      -> Running build hook: [archiso_pxe_http]
      -> Running build hook: [archiso_pxe_nfs]
      -> Running build hook: [archiso_kms]
      -> Running build hook: [block]
    ==> WARNING: Possibly missing firmware for module: wd719x
    ==> WARNING: Possibly missing firmware for module: aic94xx
      -> Running build hook: [pcmcia]
      -> Running build hook: [filesystems]
      -> Running build hook: [keyboard]
    ==> Generating module dependencies
    ==> Creating xz-compressed initcpio image: /boot/archiso.img
    ==> Image generation successful
    

    Pack the contents of squashfs-root back into a squashfs image:

    /tmp ~$ mksquashfs squashfs-root airootfs.sfs
    

    Okay, now it’s time to create the USB boot media. You will need at least a 1GB USB key for this, and you will lose all the data current on the USB key.

    If your stick was previously formatted with a FAT32 partition, skip this step:

    $ sudo fdisk /dev/sdX
    
    Welcome to fdisk (util-linux 2.27.1).
    Changes will remain in memory only, until you decide to write them.
    Be careful before using the write command.
    
    Device does not contain a recognized partition table.
    Created a new DOS disklabel with disk identifier 0xfa02f14c.
    
    Command (m for help): o
    Created a new DOS disklabel with disk identifier 0xf1b89f31.
    
    Command (m for help): n
    Partition type
       p   primary (0 primary, 0 extended, 4 free)
       e   extended (container for logical partitions)
    Select (default p): p
    Partition number (1-4, default 1): 1
    First sector (2048-2097151, default 2048): 
    Last sector, +sectors or +size{K,M,G,T,P} (2048-2097151, default 2097151): 
    
    Created a new partition 1 of type 'Linux' and of size 1023 MiB.
    
    Command (m for help): t
    Selected partition 1
    Partition type (type L to list all types): c
    Changed type of partition 'Linux' to 'W95 FAT32 (LBA)'.
    
    Command (m for help): w
    The partition table has been altered.
    Calling ioctl() to re-read partition table.
    

    Now format and mount the USB key, this will erase all data on the USB key:

    $ sudo mkfs.vfat -n ARCH_201512 /dev/sdX1
    $ mkdir /tmp/archlinux-usb
    $ sudo mount /dev/sdX1 /tmp/archlinux-usb
    

    Copy the contents of the Arch installation ISO you mounted earlier to the USB key:

    $ sudo cp -R /tmp/archlinux-iso/* /tmp/archlinux-usb/
    

    Now, we need to replace the kernel, initrd, and squashfs filesystem on the USB key with the ones we made:

    $ sudo cp /tmp/squashfs-root/boot/vmlinuz /tmp/archlinux-usb/arch/boot/x86_64/vmlinuz
    $ sudo cp /tmp/squashfs-root/boot/archiso.img /tmp/archlinux-usb/arch/boot/x86_64/archiso.img
    $ sudo cp /tmp/airootfs.sfs /tmp/archlinux-usb/arch/x86_64/airootfs.sfs
    $ cd /tmp
    /tmp ~$ echo $(md5sum airootfs.sfs) | sudo tee /tmp/archlinux-usb/arch/x86_64/airootfs.md5
    

    Feel free to delete the i686 squashfs, since we did not compile an i686 kernel:

    $ sudo rm /tmp/archlinux-usb/arch/i686/airootfs.*
    

    If you’re building the boot media yourself, you will also need to put bootia32.efi in /tmp/archlinux-usb/EFI/boot/bootia32.efi since Bay Trail tablets only have 32-bit UEFI (the CPU is 64-bit). Download bootia32.efi here.


    In summary:

    1. We downloaded Linux kernel from kernel.org
    2. We downloaded the rtl8723bs driver from GitHub
    3. We applied the patches required for SDIO from rtl8723bs to the kernel source
    4. We compiled the kernel and modules using the default Arch Linux .config file
    5. We decompressed the squashfs filesystem present on the Arch Linux ISO
    6. We installed the kernel modules compiled earlier
    7. We compiled and installed the r8723bs kernel module in the decompressed squashfs filesystem
    8. We used chroot to run depmod and generate a new initrd using mkinitcpio inside the decompressed squashfs filesystem
    9. (optional) We deleted old kernel modules from the decompressed squashfs filesystem
    10. We recompressed the squashfs filesystem
    11. We formatted our USB installation media
    12. We copied the unmodified Arch Linux ISO contents to the USB installation media
    13. We replaced vmlinuz, initrd (archiso.img). and the x86_64 compressed squashfs filesystem on the USB installation media
    14. We installed bootia32.efi on the USB installation media

    Grub

    There is an issue with the install media which I haven’t bothered to diagnose. Grub will not display the normal boot menu, so you have to type in the commands manually. You need a keyboard anyway to configure WiFi and start SSH, so you might as well get one out now…

    set root=hd0,msdos1
    linux /arch/boot/x86_64/vmlinuz archisobasedir=arch archisolabel=ARCH_201512 nomodeset
    initrd /arch/boot/x86_64/archiso.img
    boot
    

    Wireless

    If all goes well, you will have a booted tablet with a wlan0 device. Follow the Arch instructions to configure wireless.

    Or, create /etc/wpa_supplicant/MyNetwork.conf with your network details:

    ctrl_interface=/var/run/wpa_supplicant
    update_config=1
    country=US
    
    network={
      ssid="MyNetwork"
      psk="Staple Horse Battery XKCD"
    }
    

    Up the interface with wpa_supplicant:

    $ wpa_supplicant -Dnl80211 -iwlan0 -c/etc/wpa_supplicant/MyNetwork.conf
    

    If all goes well, wpa_supplicant will find and connect to your network, but you still won’t have an IP address, so switch to another TTY (e.g. ctrl+alt+F2) and run dhclient to get an IP address:

    $ dhclient wlan0
    

    Set a root password and start SSH:

    $ passwd
    $ systemctl start sshd
    

    Find the IP address of your tablet:

    $ ip addr
    

    Now you should be able to SSH to your tablet from another computer, and complete the installation (I have censored my MAC addresses):
    venue_8_pro_archiso_wlan-clean


    Notes: I haven’t actually installed Arch Linux on my Dell Venue 8 Pro (3845) yet. I need to use it over the holidays and want it to work. I will try to post a follow up in the next few months about my experience installing and using Arch Linux on it.

    Also, I did this and wrote the post in one afternoon. Usually when I post something here, I work on it for several days and then sit on the draft in case there are any mistakes. However, since I am leaving for Christmas vacation shortly, I wanted to get this out quickly so people could read it over the holidays. There may be errors or omissions in the article which prevent it from working exactly as written. If I discover any errors, I will update the article to correct them.

    Build and package your own software for OpenWRT

    Today I am going to discuss how to build and package your own software for OpenWRT.

    When I say “your own software” in this case I am referring to a C program which you want to cross-compile for the target SoC and install using the opkg package manager included in OpenWRT.

    The program I wrote is a little more complicated than your standard “Hello World” application. Here’s what I wanted to do:
    1) use libconfig to read a configuration file in /etc/config/ and then perform actions based on the configuration described in this file
    2) use sqlite3 to create a database
    3) write some meaningful data to the database

    Here’s the program flow:
    1) Open /etc/config/example-sqlite and read the values into variables
    2) Open (or create) a new SQLite3 database file at the location defined in the above configuration file
    3) Determine if the SQLite file is initialized with the target table we want to write to, and if not, create the table
    4) Write the system load average to the database
    5) Quit

    To recap, this program is different from “Hello World” in the following ways:
    1) It must read and understand a configuration file in libconfig syntax; this requires linking against the libconfig library, which we must tell opkg is a dependency
    2) It must create or open an SQLite 3 database; this requires linking against the sqlite3 library, which we must tell opkg is a depenedency
    3) It must perform some useful operations on this SQLite file

    Let’s start with compiling the C file on your native architecture. Sure, you can just use cc/gcc from bash, but this isn’t any good to OpenWRT SDK, which expects that each package will have a makefile which can be used to compile the software.

    load2sqlite.c

    #include <sys/types.h>
    #include <string.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <sqlite3.h>
    #include <libconfig.h>
    #include <sys/stat.h>
    #include <fcntl.h>
    #include <errno.h>
    
    int main(int argc, char *argv[]) {
    // ...
    

    Most importantly above, we are including sqlite3.h for SQLite support, libconfig.h, and sys/stat.h, fcntl.h,errno.h to check if the SQLite3 database file exists or not.

    You can compile this by hand quite easily, just by doing:
    cc load2sqlite.c -lsqlite3 -lconfig -o load2sqlite

    Okay, but how do we make this ready for OpenWRT SDK? By writing a makefile!

    makefile

    PROFILE = -O2 -s
    CFLAGS = $(PROFILE)
    LDFLAGS = -lsqlite3 -lconfig
    
    all: main
    
    # build it
    main:
    	$(CC) $(CFLAGS) load2sqlite.c $(LDFLAGS) -o load2sqlite
    
    # clean it
    clean:
    	rm load2sqlite
    

    Okay, so now if you type make in the directory, magically you will end up with an executable called load2sqlite!

    But, this is a native binary, and it’s somewhat unlikely that your OpenWRT device is on the same architecture.

    load2sqlite: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=9661b88e92b553d0556cbeeafccf04d2526c770f, stripped
    

    If you run it, you’ll see that it looks for the sqlite database file, can’t find it, and so initalizes a new one with the “readings” table.

    [[email protected] src]$ ./load2sqlite 
    Database file /tmp/sqlite3.db does not exist
    Initialized database with readings table
    [[email protected] src]$ echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
    2015-10-28 22:48:42|0.57|0.56|0.57
    

    And if you run it again, without removing the SQLite3 file that was created, you’ll see this output:

    [[email protected] src]$ ./load2sqlite
    SQLite database opened
    Found readings table
    [[email protected] src]$ echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
    2015-10-28 22:48:42|0.57|0.56|0.57
    2015-10-28 22:49:00|0.47|0.54|0.57
    

    Before we proceed further, I want to show you the directory structure so you have an idea of where we just were when we did this compilation. We are currently in the the src directory.

    load2sqlite/
    |-- Makefile
    |-- README
    `-- src
        |-- load2sqlite.c
        |-- load2sqlite.conf
        `-- makefile
    

    Now let’s move up to the load2sqlite directory and work on the OpenWRT Makefile (seen above).

    Here is the complete file, and then we will discuss it section by section:
    Makefile

    #
    # Copyright (C) 2006-2015 OpenWrt.org
    #
    # This is free software, licensed under the GNU General Public License v2.
    # See /LICENSE for more information.
    #
    
    include $(TOPDIR)/rules.mk
    
    PKG_NAME:=load2sqlite
    PKG_VERSION:=1.0.1
    PKG_RELEASE:=5
    PKG_MAINTAINER:=Hal Martin 
    PKG_LICENSE:=GPL-2
    PKG_CONFIG_DEPENDS:=libsqlite3 libconfig
    
    include $(INCLUDE_DIR)/package.mk
    
    PKG_BUILD_DIR := $(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION)
    
    TARGET_LDFLAGS+= \
      -Wl,-rpath-link=$(STAGING_DIR)/usr/lib \
      -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/libconfig/lib \
      -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/sqlite/lib
    
    define Package/load2sqlite
      SECTION:=utils
      CATEGORY:=Utilities
      DEPENDS:=+libsqlite3 +libconfig
      TITLE:=SQLite example program, creates or opens a user defined SQLite database
      URL:=https://github.com/halmartin/load2sqlite
      MENU:=1
    endef
    
    define Package/load2sqlite/description
     Example SQLite is a sample program built using libsqlite3 and libconfig
     which creates or opens a user-defined SQLite3 database and performs some
     simple verification checks on the file to ensure that the target table (readings)
     exists, and if not creates the table, then inserts a row with the current system
     time, and the load (1 minute, 5 minute, 15 minute).
    endef
    
    define Build/Prepare
    	mkdir -p $(PKG_BUILD_DIR)
    	$(CP) ./src/* $(PKG_BUILD_DIR)/
    endef
    
    define Build/Configure
    endef
    
    define Build/Compile
    	$(MAKE) -C $(PKG_BUILD_DIR) $(TARGET_CONFIGURE_OPTS)
    endef
    
    define Package/load2sqlite/install
    	$(INSTALL_DIR) $(1)/bin
    	$(INSTALL_BIN) $(PKG_BUILD_DIR)/load2sqlite $(1)/bin/
    	$(INSTALL_DIR) $(1)/etc/config
    	$(INSTALL_CONF) $(PKG_BUILD_DIR)/load2sqlite.conf $(1)/etc/config/load2sqlite
    endef
    
    $(eval $(call BuildPackage,load2sqlite))
    

    If you clone the OpenWRT source and take a look at basically any package, you’ll see a Makefile that looks similar to the one above.

    Let’s look at the package information:

    PKG_NAME:=load2sqlite
    PKG_VERSION:=1.0.1
    PKG_RELEASE:=5
    PKG_MAINTAINER:=Hal Martin 
    PKG_LICENSE:=GPL-2
    

    Here is where we define core details of our package, such as the name (e.g. what opkg will know it as), the version (useful for upgrading later), maintainer, and license.

    TARGET_LDFLAGS+= \
      -Wl,-rpath-link=$(STAGING_DIR)/usr/lib \
      -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/libconfig/lib \
      -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/sqlite/lib
    

    Since we want to build a program which links against external libraries, we must also tell the compiler where to find the header files for these libraries, so that the linking process does not fail during compilation. Above you can see that we are linking to libconfig and sqlite libraries.

    define Package/load2sqlite
      SECTION:=utils
      CATEGORY:=Utilities
      DEPENDS:=+libsqlite3 +libconfig
      TITLE:=SQLite example program, creates or opens a user defined SQLite database
      URL:=https://github.com/halmartin/load2sqlite
      MENU:=1
    endef
    

    This is where you define the package for the OpenWRT build system and declare things like dependencies, and the description that will be present when you run menuconfig (which is how you will select your package to be built as part of an image).

    Without declaring dependencies, you may find that you can build, package, and install your software, but it won’t run! So, by declaring the dependencies (packages which provide the libraries we link against) we ensure that when we type opkg install load2sqlite and libconfig and libsqlite3 are not installed, opkg knows to go and install them before installing our program. Now we can safely run the program because all the required libraries are installed on the device!

    define Build/Prepare
    	mkdir -p $(PKG_BUILD_DIR)
    	$(CP) ./src/* $(PKG_BUILD_DIR)/
    endef
    
    define Build/Configure
    endef
    

    Since our utility is quite simple, as *NIX software goes, the preparation steps are to create the build directory and copy the source from the source directory to the build directory. Since there is nothing to configure in our sample program, the configure step is empty (otherwise the OpenWRT build system will attempt to configure the package and fail because we haven’t bothered to implement this).

    define Build/Compile
    	$(MAKE) -C $(PKG_BUILD_DIR) $(TARGET_CONFIGURE_OPTS)
    endef
    
    define Package/load2sqlite/install
    	$(INSTALL_DIR) $(1)/bin
    	$(INSTALL_BIN) $(PKG_BUILD_DIR)/load2sqlite $(1)/bin/
    	$(INSTALL_DIR) $(1)/etc/config
    	$(INSTALL_CONF) $(PKG_BUILD_DIR)/load2sqlite.conf $(1)/etc/config/load2sqlite
    endef
    

    Finally, compile and install the software. As you can see above, I didn’t include an install directive in the makefile of the application, it is instead done manually within the OpenWRT Makefile. This is your choice, since I was designing this program specifically to run on OpenWRT, I saw no need to incorporate the installation steps in the makefile of the program.

    And, finally:

    $(eval $(call BuildPackage,load2sqlite))
    

    This line is required for OpenWRT to build the package. Forget this line, and you will sit there wondering why your package is not being built!


    Okay, now we have prepared our software to be built for OpenWRT. It would be stupid of me to get this far and not tell you how to compile it using the OpenWRT toolchain!

    Following the excellent OpenWRT documentation, we need to set up a buildroot.

    Install the dependencies (instructions for Debian/Ubuntu):

    sudo apt-get install git-core build-essential libssl-dev libncurses5-dev unzip subversion mercurial
    

    Clone the OpenWRT Chaos Calmer release:

    git clone git://git.openwrt.org/15.05/openwrt.git
    

    I find that the stock OpenWRT repository is a bit light on some of the software I like to have on my routers, so I take step 3 and install the additional feeds:

    cd openwrt
    ./scripts/feeds update -a
    ./scripts/feeds install -a
    

    Follow step 4 to ensure you have all the required dependencies installed on your host system!

    make defconfig
    make prereq
    # don't forget to copy load2sqlite to package/utils/ before running this step, or the package won't appear in the menu!
    make menuconfig
    

    If everything has gone well thus far (e.g. no errors in the OpenWRT Makefile, and you put load2sqlite in package/utils/ then you should see the following in your menuconfig:

    menuconfig_load2sqlite

    menuconfig_load2sqlite_desc

    Now I already have an official OpenWRT build installed on my router, so I don’t need to build an entire image, just the package I want to install. To do this, we must first build the cross compilation toolchain required to compile for a different CPU architecture.

    Warning: the OpenWRT buildroot is around 6GB on disk, so ensure you have the necessary space before starting!

    make tools/install
    # this will take a while the first time
    make toolchain/install
    # this will also take a while the first time
    

    When we have the tools and toolchain compiled, we can compile our package:

    make package/load2sqlite/compile
    

    This will create an ipkg file in bin/ramips/packages/base/load2sqlite_1.0.1-5_ramips_24kec.ipk which we need to copy to our router to install:

    scp bin/ramips/packages/base/load2sqlite_1.0.1-5_ramips_24kec.ipk [email protected]:/tmp/
    # scp completes
    ssh [email protected]
    [email protected]'s password:
    
    BusyBox v1.23.2 (2015-07-25 03:03:02 CEST) built-in shell (ash)
    
      _______                     ________        __
     |       |.-----.-----.-----.|  |  |  |.----.|  |_
     |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
     |_______||   __|_____|__|__||________||__|  |____|
              |__| W I R E L E S S   F R E E D O M
     -----------------------------------------------------
     CHAOS CALMER (15.05, r46767)
     -----------------------------------------------------
      * 1 1/2 oz Gin            Shake with a glassful
      * 1/4 oz Triple Sec       of broken ice and pour
      * 3/4 oz Lime Juice       unstrained into a goblet.
      * 1 1/2 oz Orange Juice
      * 1 tsp. Grenadine Syrup
     -----------------------------------------------------
    [email protected]:~# opkg install /tmp/load2sqlite_1.0.1-5_ramips_24kec.ipk 
    Installing load2sqlite (1.0.1-4) to root...
    Installing libsqlite3 (3081101-1) to root...
    Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/packages/libsqlite3_3081101-1_ramips_24kec.ipk.
    Installing libpthread (0.9.33.2-1) to root...
    Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/base/libpthread_0.9.33.2-1_ramips_24kec.ipk.
    Installing libconfig (1.4.9-1) to root...
    Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/base/libconfig_1.4.9-1_ramips_24kec.ipk.
    Configuring libpthread.
    Configuring libconfig.
    Configuring libsqlite3.
    Configuring load2sqlite.
    

    Now that our package is installed, we can test it!

    [email protected]:~# /bin/load2sqlite 
    Database file /tmp/sqlite3.db does not exist
    Initialized database with readings table
    

    If you install sqlite3-cli we can inspect the row added to the file:

    [email protected]:~# opkg install sqlite3-cli
    [email protected]:~# echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
    2015-10-31 20:47:33|0.76|0.4|0.25
    

    Since this is just an example program, it is one-shot (e.g. not a daemon). If you really do want to track the load of our OpenWRT router, just add /bin/load2sqlite to crontab (e.g. every hour) and you’ll have this tracking info in the SQLite database.

    If you run it multiple times, you get another row added to the file each time the program is run:

    [email protected]:~# /bin/load2sqlite 
    SQLite database opened
    Found readings table
    [email protected]:~# echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 
    2015-10-31 20:47:33|0.76|0.4|0.25
    2015-10-31 20:53:43|0.02|0.2|0.22
    2015-10-31 21:23:19|0.08|0.04|0.05
    

    Note that by default the file is saved to /tmp/, which on OpenWRT is a ramdisk. This means that the file will be lost when you reboot, or if you leave it running unattended for too long, the file size will grow to the point where the ramdisk will consume all available memory and the router will crash. For this reason, I suggest you modify the configuration file /etc/config/load2sqlite to point to non-volatile storage (such as a USB stick).


    Source code: https://github.com/halmartin/load2sqlite


    Why write another OpenWRT software guide?

    Well, while I was looking for resources on how to build and package software for OpenWRT, I ran into a lot of posts about people compiling simple “Hello World” programs for OpenWRT, but for my particular use case, I wanted to utilize multiple libraries in my program, and I couldn’t find any good instructions on how to compile a program with linked libraries for OpenWRT.

    Disclaimer: I’m not a C expert, so maybe there are some headers there which are not strictly necessary, but it works for me and the executable size is quite small.

    If you wish to further reduce the size of your executable, you can tell the compiler to strip it of the symbol table and relocation information. Do this by appending -s to the PROFILE line in the makefile. When I did this on my laptop, the output went from 9.9KB to 7.0KB, or a savings of 30%

    I have tested this on Chaos Calmer (15.05), and I expect the instructions would also work on Barrier Breaker (14.07) however I didn’t try this, so I cannot say certainly that it will work.

    Building Linux 4.1 for the Banana Pi

    This post is a follow up to my original post Building BananaPi LeMaker Kernel.

    If you’re just looking for a vanilla Debian or Ubuntu image for your Banana Pi that utilizes a Linux kernel newer than 3.4.xxx, then stop reading and go to this page maintained by Igor Pečovnik. He provides pre-built Debian and Ubuntu images for a variety of Banana Pi boards.

    If you want to manually build an image, he has put the build scripts he uses up on his GitHub repository. While I tried to do everything manually starting from my last post, I ended up building a kernel that would not boot. So I shamelessly stole the kernel configuration from Igor, and the resulting kernel boots.

    The GMAC driver which required so much patching for the 3.4 kernel was mainlined in 3.17. As such, these instructions should work for any kernel newer than 3.17. I am building 4.1.3 in my script.

    Here is the Jenkins/bash script to build the kernel, modules, and boot goodness you need (a direct link to the .sh file is at the end of the post):

    if [ ! -d "linux-4.1.3" ]; then
    wget https://www.kernel.org/pub/linux/kernel/v4.x/linux-4.1.3.tar.xz
    tar -Jxvf linux-4.1.3.tar.xz
    fi
    cd linux-4.1.3
    wget https://watchmysys.com/blog/wp-content/uploads/2015/07/banana-pi-linux-4.1.3-config.txt -O .config
    make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- clean
    make -j4 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- LOADADDR=0x40008000 zImage dtbs
    make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=output modules
    make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=output modules_install
    mkdir -p output/boot/
    cp arch/arm/boot/zImage output/boot/
    cp arch/arm/boot/dts/sun7i-a20-bananapi.dtb output/boot/
    cat > output/boot/boot.cmd < output/boot/uEnv.txt << EOF
    fatload mmc 0 0x46000000 zImage
    fatload mmc 0 0x49000000 sun7i-a20-bananapi.dtb
    setenv bootargs console=ttyS0,115200 [earlyprintk] root=/dev/mmcblk0p2 rootwait panic=10 rootfstype=ext4 rw ${extra}
    bootz 0x46000000 - 0x49000000
    EOF
    mkimage -C none -A arm -T script -d output/boot/boot.cmd output/boot/boot.scr
    cd ..
    if [ ! -d "u-boot-2015.04" ]; then
    wget ftp://ftp.denx.de/pub/u-boot/u-boot-2015.04.tar.bz2
    tar -jxvf u-boot-2015.04.tar.bz2
    fi
    cd u-boot-2015.04
    make -s CROSS_COMPILE=arm-linux-gnueabihf- clean
    make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- Bananapi_defconfig
    make -j4 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
    cp u-boot-sunxi-with-spl.bin ../linux-4.1.3/output/boot/
    cd ../linux-4.1.3/
    cat > output/boot/uEnv.txt << EOF
    fatload mmc 0 0x46000000 zImage
    fatload mmc 0 0x49000000 sun7i-a20-bananapi.dtb
    setenv bootargs console=ttyS0,115200 [earlyprintk] root=/dev/mmcblk0p2 rootwait panic=10 rootfstype=ext4 rw ${extra}
    bootz 0x46000000 - 0x49000000
    EOF
    tar -C output -cjvf ../linux-bananapi-4.1.3.tar.bz2 boot/ lib/
    

    Here is what the partition layout of my SDHC card:

    [email protected]:~# fdisk -l /dev/mmcblk0
    
    Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
    4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000
    
            Device Boot      Start         End      Blocks   Id  System
    /dev/mmcblk0p1   *        2048      133119       65536    c  W95 FAT32 (LBA)
    /dev/mmcblk0p2          133120    15523839     7695360   83  Linux
    

    Here is the contents of the boot partition (mmcblk0p1, vfat):

    [email protected]:~# ls /boot
    boot.cmd  boot.scr  sun7i-a20-bananapi.dtb  uEnv.txt  zImage
    

    boot.cmd

    fatload mmc 0 0x46000000 zImage
    fatload mmc 0 0x49000000 sun7i-a20-bananapi.dtb
    setenv bootargs console=ttyS0,115200 earlyprintk root=/dev/mmcblk0p2 rw rootwait panic=10 
    bootm 0x46000000 - 0x49000000
    

    uEnv.txt

    fatload mmc 0 0x46000000 zImage
    fatload mmc 0 0x49000000 sun7i-a20-bananapi.dtb
    setenv bootargs console=ttyS0,115200 [earlyprintk] root=/dev/mmcblk0p2 rootwait panic=10 rootfstype=ext4 rw ${extra}
    bootz 0x46000000 - 0x49000000
    

    You will need to update u-boot on the SD card to v2015.04. If you use the script I provide above, this file is in boot/u-boot-sunxi-with-spl.bin. You need to write it to the SD card using dd:

    dd if=boot/u-boot-sunxi-with-spl.bin of=/dev/mmcblk0 bs=1024 seek=8

    More notes:
    I was unable to build this kernel running Debian 7 (Wheezy) because binutils is too old. Unfortunately the official repositories do not have a newer version available for Wheezy (slash) I was too lazy to look for a repository that might have a newer version. As such, I upgraded by Jenkins build box to Debian 8 (Jessie) to build this kernel.

    Additionally, I had to upgrade from a 1GB SD Card in my Banana Pi to an 8GB SDHC Card because the new u-boot does not seem to like small (non-SDHC) cards.

    Banana Pi info:

    [email protected]:~# free -m
                 total       used       free     shared    buffers     cached
    Mem:           996         72        923          0          4         19
    -/+ buffers/cache:         48        948
    Swap:            0          0          0
    [email protected]:~# uname -a
    Linux bpi 4.1.3-bananapi #2 SMP Sun Jul 26 15:41:54 CEST 2015 armv7l GNU/Linux
    [email protected]:~# lsmod
    Module                  Size  Used by
    [email protected]:~#
    

    Speed test of the network interface:

    # Laptop with Gigabit wired connection, saving to /tmp/ ramdisk
    [[email protected] ~]$ nc -lp 5000 | dd of=/tmp/zerofile
    # Banana Pi
    [email protected]:~# dd if=/dev/zero bs=1M count=2000 | nc laptop 5000
    2000+0 records in
    2000+0 records out
    2097152000 bytes (2.1 GB) copied, 73.5536 s, 28.5 MB/s
    

    Yes, it is a Gigabit link, but not the fastest. It does seem to be quite stable, and since I am not using my Banana Pi for a bandwidth intensive purpose, this speed is fine for me.

    The build script does everything with relative paths, and can be run as a normal user. The output is a tar.bz2 archive containing u-boot binary, boot folder, and kernel modules. You will need sudo/root to install the u-boot bin file with dd as described above.

    Download kernel .config: here
    Download build script: here
    Download u-boot 2015.04: here (SHA1SUM: 8bf4f738ba8aa18ab5d45fca324587f0749f7c10)
    Download tar archive with u-boot, kernel, and modules: here (SHA1SUM: d94a8da66ce6d77a1ceb4569740cadf2c8c67e72)

    From Amazon with love: not quite 32GB micro SD cards

    I needed more space in my tablet and phone, so I went to everyone’s favourite online merchant, Amazon, and purchased two SanDisk 32GB Class 10 micro SDHC cards.

    Now, if you haven’t bought SanDisk micro SD cards lately, let me warn you now, the silk screen looks horrible. At first I thought they were counterfeit, it’s that bad. I wonder if the guys at SanDisk did some research to see how shitty they could make the silk screen and not have it appear in cell phone photos. My potato camera phone has just enough noise that it’s difficult to tell just how bad the SanDisk logo looks, but it’s awful.

    My camera phone isn't the greatest, but these cards look just blurry and noisy in reality

    My camera phone isn’t the greatest, but these cards look just as blurry and noisy in reality

    But a bad silk screen can be overlooked if the cards themselves still function, which brings me to my next point:

    [38853.623229] scsi 9:0:0:0: Direct-Access     Generic- SD/MMC           1.00 PQ: 0 ANSI: 0 CCS
    [38854.311664] sd 9:0:0:0: [sdb] 60367872 512-byte logical blocks: (30.9 GB/28.7 GiB)
    [38854.312801] sd 9:0:0:0: [sdb] Write Protect is off
    [38854.312812] sd 9:0:0:0: [sdb] Mode Sense: 03 00 00 00
    [38854.313898] sd 9:0:0:0: [sdb] No Caching mode page found
    [38854.313917] sd 9:0:0:0: [sdb] Assuming drive cache: write through
    [38854.329950]  sdb: sdb1
    [38854.333772] sd 9:0:0:0: [sdb] Attached SCSI removable disk
    
    Disk /dev/sdb: 28.8 GiB, 30908350464 bytes, 60367872 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x00000000
    
    Device     Boot Start      End  Sectors  Size Id Type
    /dev/sdb1        8192 60367871 60359680 28.8G  c W95 FAT32 (LBA)
    

    28.7GiB?! What the hell? I know storage manufacturers have redefined a gigabyte to be 1,000,000,000 bytes (1 billion bytes) instead of 1,073,741,824 bytes (1024^3) but this is some next level math happening here.

    For comparison, here is a Transcend 32GB Class 10 micro SDHC card:

    [38782.491351] scsi 7:0:0:0: Direct-Access     Generic- SD/MMC           1.00 PQ: 0 ANSI: 0 CCS
    [38783.265974] sd 7:0:0:0: [sdb] 61831168 512-byte logical blocks: (31.6 GB/29.4 GiB)
    [38783.267091] sd 7:0:0:0: [sdb] Write Protect is off
    [38783.267099] sd 7:0:0:0: [sdb] Mode Sense: 03 00 00 00
    [38783.268192] sd 7:0:0:0: [sdb] No Caching mode page found
    [38783.268197] sd 7:0:0:0: [sdb] Assuming drive cache: write through
    [38783.273707]  sdb: sdb1
    [38783.277183] sd 7:0:0:0: [sdb] Attached SCSI removable disk
    
    Disk /dev/sdb: 29.5 GiB, 31657558016 bytes, 61831168 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x000db221
    
    Device     Boot Start      End  Sectors  Size Id Type
    /dev/sdb1        2048 61831167 61829120 29.5G  c W95 FAT32 (LBA)
    

    29.4GiB is a lot closer to what we expect from a 32GB (32,000,000,000 byte) card. The precise number is 29.80232GiB, but okay, it’s not much less.

    So the SanDisk card is a about 700MB smaller than the Transcend. That alone was enough to make me apply for an RMA…

    but-wait-theres-myrrh

    A Class 10 SD card is defined as having at least 10MB/s sequential write performance. Since we already know these cards over-advertise their capacity, what is the performance like?

    Card 1

    Card 1

    Card 1 makes it past the Class 10 specification, averaging a sequential write speed of 11.7MB/s using H2testw with a fresh FAT filesystem. Not great, but at least it’s within specifications.

    Card 2

    Card 2

    Card 2 however, barely makes it past Class 4. With an average write speed of only 5.55MB/s, this card is just abysmal.

    What about read speeds? Well, Amazon claims up to 48MB/s (megabytes per second) reading speed:

    sandisk_48MBs_read

    Card 1 managed a semi-respectable 18.2MB/s read speed. Card 2 however, was just awful and couldn’t give more than 7.2MB/s read speed.

    Verdict:
    Back to Amazon you go, shitty SanDisk SD cards. Next time I am going to buy Transcend, and from now on I’ll be testing any cards I buy to make sure they:
    A) Are actually the advertised capacity
    B) Meet the minimum specifications for their advertised Class
    C) Can be read in all my devices, unlike these cards which wouldn’t read at all in my MacBook Pro

    Testing method:
    Both cards were tested in a Dell Venue 8 Pro running Windows 8.1.

    I would have liked to have tested their raw speed using dd in Linux, but unfortunately my MacBook Pro would not read the cards at all! dmesg was full of SD sector and command errors when I put the cards in the reader.

    Edit (26.07.2015): I bought Transcend cards from Amazon, and again they were not true 32GB cards. My old Transcend card was made in Taiwan, whereas all the “32GB” cards I have gotten from Amazon this year are made in China.

    It makes me wonder if SanDisk and Transcend have licensed an ODM to produce cards for them, and then stuck their silk screen on the cards. I suspect this because both the SanDisk and Transcend cards I received identified themselves as SL32G cards, whereas my Taiwan manufactured Transcend identifies as USD.

    For the time being, I have ceased to purchase 32GB microSDHC cards from Amazon until I can find a brand that sells a card that is actually 32GB.

    From SATURN with love: a bad USB power supply

    Recently I needed another USB charger for my devices. I went to Saturn and picked up this Innergie adapter, which is rated for 10W (2A at 5V).

    innergie_front

    innergie_rear

    But, it doesn’t work. Sure, it can put out 2.5A, if you short it. I managed to get 0.6A out of it charging a 24Wh lithium-polymer battery, but at that current the voltage drops to 4.5V, which is outside of the USB specification and will not charge a mobile phone or tablet.

    innergie_fluke_voltage_current

    When I did plug in a mobile phone, it drew about 0.3A (1.5W) but the charging light did not turn on. The Innergie cannot even hold up at 1.5W, it dropped to 4.75V!

    I lost the receipt, otherwise I would return it for my money back. So into the junk pile this goes… Lesson learned, next time save the receipt!

    Installing CentOS 7 with a chroot

    I needed to install CentOS 7 on an embedded PC with UEFI and 2 SSD disks in mdadm RAID1.

    While I’m sure the guys at Red Hat work very hard on CentOS, the installer is a piece of cr*p, especially when it comes to disk partitioning. I have never hated any installer more than the CentOS disk partitioner. I don’t know what happened. The disk partitioning tool in CentOS 6 installer was fine, I had no problems using it, but in the 7 installer it’s just a nightmare to do anything. In my opinion Windows installer does a better job of disk partitioning than the CentOS 7 installer.

    While many people who like CentOS will proclaim that the error is between the keyboard and chair, I welcome them to provide a write-up and screenshots of how to accomplish my desired partitioning scheme using the graphical installer. If the instructions are shorter than this blog post, next time I won’t use a chroot to install.

    I loosely based my method off of this post, but immediately found I had to deviate because I didn’t have USB install media with all the required commands on it.

    Required tools:
    1) USB stick with some Linux distro on it (I prefer the Gentoo minimal installer, it’s small and it includes lots of useful utilities)
    2) USB stick with the CentOS minimal installer on it
    3) About 2GB of free space, you can use a ramdisk, or create an extra partition using the free space on the USB sticks (CentOS occupies about 800MB, Gentoo about 300MB)

    Steps:
    1) Boot the Gentoo installer off the USB stick
    2) Partition your disks however you like using gdisk, fdisk, or parted
    3) Create your mdadm array(s)
    4) Plug in the CentOS 7 USB stick and mount it to a temporary mount point (e.g. /tmp/centos)

    Inside the CentOS USB stick you will find LiveOS/squashfs.img, you need to loopback mount this:

    livecd ~ # mount -o loop /tmp/cinstall/LiveOS/squashfs.img /tmp/csquashfs/

    Now we have yet another image to mount, this one within the squashfs image:

    livecd ~ # mount -o loop /tmp/csquashfs/LiveOS/rootfs.img /tmp/croot

    Finally we have a Linux filesystem. But unfortunately we cannot use it for anything as it is mounted read-only and there is no resolv.conf present, so no domain names can be resolved. This is why you need ~2GB of free space somewhere (or 4GB of RAM).

    livecd ~ # mkfs -q /dev/ram1 1572864
    livecd ~ # mkdir -p /tmp/ramdisk
    livecd ~ # mount /dev/ram1 /tmp/ramdisk
    livecd ~ # rsync -avHp /tmp/croot /tmp/ramdisk
    

    Now that we have the installer rootfs somewhere writable, copy /etc/resolv.conf to the filesystem:

    livecd ~ # cp /etc/resolv.conf /tmp/ramdisk/etc/resolv.conf

    Mount your destination partition for CentOS somewhere you can access from within the chroot:

    livecd ~ # mount /dev/vg0/centos /tmp/ramdisk/mnt

    Chroot to the installer environment:

    livecd ~ # chroot /tmp/ramdisk

    Download the CentOS release RPM and install it to the destination partition:

    bash-4.2# wget http://mirror.centos.org/centos/7/os/x86_64/Packages/centos-release-7-0.1406.el7.centos.2.3.x86_64.rpm
    bash-4.2# rpm --root=/mnt --nodeps -i centos-release-7-0.1406.el7.centos.2.3.x86_64.rpm

    Because yum is missing the yummain module in the installation environment, we need to download and install the yum RPM on the installer partition:

    bash-4.2# wget http://mirror.centos.org/centos/7/os/x86_64/Packages/yum-3.4.3-118.el7.centos.noarch.rpm
    bash-4.2# rpm -i --nodeps yum-3.4.3-118.el7.centos.noarch.rpm

    Now finally we can run yum on the destination partition to install CentOS:

    bash-4.2# yum --installroot=/mnt update
    bash-4.2# yum --installroot=/mnt install -y yum
    bash-4.2# yum --installroot=/mnt install -y @core kernel
    bash-4.2# yum --installroot=/mnt install -y grub2-efi efibootmgr lvm2 mdadm \
    dosfstools kernel

    Now unfortunately I hit a small snag: the Gentoo installer isn’t EFI aware. Exit the chroot, but remember to copy /etc/resolv.conf to the destination partition:

    livecd ~ # cp /etc/resolv.conf /tmp/ramdisk/mnt/etc/

    Poweroff and unplug the Gentoo installer USB stick. Plug in the CentOS installer stick and boot to the rescue environment. Skip rootfs detection.

    Mount the partition/LV slice containing your CentOS installation:

    sh-4.2# mkdir /mnt/centos
    sh-4.2# mount /dev/vg0/centos /mnt/centos
    sh-4.2# mount -t proc proc /mnt/centos/proc
    sh-4.2# mount --rbind /dev /mnt/centos/dev
    sh-4.2# mount --rbind /sys /mnt/centos/sys
    sh-4.2# chroot /mnt/centos /bin/bash

    Save the mdadm array information to the mdadm.conf configuration file:

    bash-4.2# mdadm --detail --scan > /etc/mdadm.conf

    Format your EFI boot partition:

    bash-4.2# mkfs.vfat /dev/sda1
    bash-4.2# mkdir /boot/efi
    bash-4.2# mount /dev/sda1 /boot/efi

    Install grub:

    bash-4.2# grub2-install
    Installing for x86_64-efi platform
    ...
    Installation finished. No error reported.

    Now, dear reader, this is the part where you do not see the hours I spent debugging why dracut would not find my root partition (hint: see Fedora wiki for dracut debugging steps). The tl;dr is that the mdadm array wasn’t being assembled, for reasons still unknown. To solve this we need to add our array UUID as an additional kernel parameter for grub:

    bash-4.2# MD_UUID=$(mdadm -D /dev/md0 | grep UUID | awk '{print $3}')
    bash-4.2# grubby --update-kernel=/boot/vmlinuz-3.10.0-123.20.1.el7.x86_64 \
    --args="rd_MD_UUID=$MD_UUID"
    bash-4.2# grub2-mkconfig -o /boot/grub2/grub.cfg

    Absolutely verify in /boot/grub2/grub.cfg that the correct rd_MD_UUID was appended to linuxefi, or like me, you may be left wondering why your system won’t boot.

    Check with efibootmgr that a menu entry was created:

    bash-4.2# efibootmgr -v
    ...
    Boot0014* grub  HD(1,800,32000,SUPER-LONG-UUID)File(\EFI\grub\grubx64.efi)

    Don’t forget to set a root password:

    bash-4.2# passwd
    Changing password for user root.
    New password:
    Retype new password:
    passwd: all authentication tokens updated successfully

    Configuring the hostname (using hostnamectl, or editing /etc/sysconfig/network and /etc/hostname), udev rules for eth* interface names, and static network configuration in /etc/sysconfig/network-scripts/ifcfg-eth* is left as an exercise for the reader.

    After rebooting

    If when you reboot you find that you cannot login as root using the password you specified, it’s probably SELinux. Normally I hate disabling SELinux, but in this case I was so tired of spending a day and a half debugging booting issues, I just disabled it and went on with the setup.

    Despite what the wonderful CentOS installer tells you, you do not require a separate /boot partition. Right now this is what the partition layout looks like:

    Disk /dev/sda: 128.0 GB, 128035676160 bytes, 250069680 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 4096 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disk label type: gpt
    
    
    #         Start          End    Size  Type            Name
     1         2048       206847    100M  EFI System      EFI System
     2       206848    250069646  119.1G  Linux RAID      Linux RAID

    Because the contents of /boot/efi is static, and cannot be mdraid, remember to copy the contents of /dev/sda1 to /dev/sdb1 so that if your first drive ever dies, you will still have the required EFI components to boot off the second drive.

    Overall, I can’t say this was a lot of fun. But I did learn a lot more about dracut and the CentOS booting process, and I can still say with confidence that I hate the CentOS 7 installer.