Author Archives: Hal Martin

About Hal Martin

In my free time I like experiment with hardware and embedded systems. Here I write about personal projects and random adventures into firmware land.

Intel DK200 IoT Gateway

Earlier this year I was at a conference and heard from other attendees that the Intel booth was giving away IoT gateways. Never one to turn down free conference swag, I hurried over to the Intel booth and was told to pick up a gateway out of a pallet of boxes just delivered (and rapidly disappearing).

The Intel IoT gateway series is codenamed Moon Island, but the design targeting the transportation market is codenamed Clanton Hill. Clanton Hill known to us mortals as the “Intel DK200 Series Gateway Solution for the Internet of Things (IoT)” quite the mouthful.

Let’s get down to it.

Availability
Unless you happen to be at a conference where Intel reps are handing these out like candy, I don’t think it’s practical to try and buy one yourself:

dk200_mouser

Some interesting details to note about this product:

  1. Although released in 2014, the DK200 still costs more than the new MacBook (3,712.50 EUR versus 3,199 EUR)
  2. It’s End Of Life

When a low volume product goes EOL and you still have stock, I guess giving it away at conferences is the next logical step.

Hardware Specifications
The DK200 (datasheet) is targeted toward the transportation industry, and it really shows in the appearance of the device:

Only available in 'Cosmic Black'

Only available in ‘Cosmic Black’

I don’t work in the transportation industry, and have never seen connectors that look like this before. They’re very well made, and I suspect probably do a good job of keeping dust, dirt, and debris out of the ports. Since I don’t wish to make a mess by throwing dirt and debris at it, I’m going to have to trust the engineers who designed it.

The build quality is quite good, as one might expect from a device selling for 3,700 EUR. Nearly every screw is secured with loctite to prevent vibration from loosening them:

DK200 screw with loctite

No pentalobe nonsense here

However, I was surprised to find that despite all the physical hardening applied to the enclosure, I couldn’t find any information on an IP rating. In fact the top and bottom of the case don’t appear to offer any additional dust or water seal. There’s clearly been a lot of thought put into the design of this enclosure to withstand vibration and dirt, so it’s strange that there doesn’t seem to be water protection of any kind.

Processor
The Intel Quark series SoC was introduced in late 2013. The X1020D in the DK200 is a single core SoC based around a 80486 core running at 400MHz, with modern I/O and memory.

dk200_x1020d

In 2014 a leaked product roadmap suggested a successor to the X1000 series named “Dublin Bay” to be released in 2015. Then news emerged that “Dublin Bay” had been cancelled, to be replaced by “Liffy Island” and “Seal Beach” which would be released in 2015. As of late 2016 Intel has not released a direct successor to the X1000 series, and there is no new news of “Liffy Island” or “Seal Beach” being cancelled (or released). So it’s anyone’s guess whether Intel is still even interested in the IoT gateway market.

Storage
The DK200 doesn’t include any of the typical storage buses like SATA, NVMe, or NAND (EMMC). This is not overly surprising given the embedded nature of the hardware (requiring lower power) and the simplicity of the Quark processor.

The only storage option is a micro SDHC card, and the DK200 includes an 8GB class 4 micro SD card:
dk200_sdhc

Given that it’s a class 4 card, the performance is quite poor. Use of an SD card for storage isn’t a bad decision per se, but the DK200 uses ext3 for the root partition. Ext3 is not a flash aware filesystem. SD cards have only basic wear leveling, and ext3 has no wear leveling. So it hardly seems like the appropriate combination of storage and filesystem for a headless embedded device with an expected lifetime of 5-10 years.

Input and output

  • Dual 100Mbit Ethernet controllers
  • 3 x USB 2.0 host and 1x device
  • Audio in/out
  • CAN bus
  • RS-232
  • GPIO
  • 1x half-height mini PCI-e slot (populated with Intel 7260)
  • 1x full-height mini PCI-e slot (unpopulated; for 3G modem/GPS)

The Intel documentation also mentions ZigBee, however this is an external device, presumably attached via the USB bus.

Power consumption
Development platforms aren’t known for being highly optimised devices. They often include extra I/O which would not necesssarily be included in the final product, and as such do not have the same energy efficiency as a finished product.

This being said, I was quite surprised that a device intended for 24/7 operation in an embedded environment, and especially serving the “Internet of Things” market, could be so energy inefficient. Issuing a poweroff command in Linux results in the platform going into an S5 (shutdown) state. I was surprised to discover that the energy consumption in the S5 state is 2W. This seems quite high for a device which includes an ignition input for automatic power-on and shutdown.

When booting, the device peaks at 7.9W consumption, while the idle power consumption is 7.5W. This is almost certainly due to the added peripherals as the TDP of the Quark processor is only 2W.

It’s difficult to see how Intel expects the Quark platform to compete with ARM. My PandaBoard ES, an ARM-based development board from 2011, peaks at 4W, idles at 2W, and draws nothing when off. Now some might argue that comparing an ARM board from 2011 with an Intel IoT gateway from 2014 isn’t valid, but they do have a lot of similar features. Now, I will grant that the PandaBoard is not in a rugged enclosure with fancy connectors, but since it cost 95% less than the DK200 does, there’s some room in the budget for an enclosure and funky connectors. And, since Texas Instruments has stopped making OMAP chips, the PandaBoard gets about the same amount of vendor support as the DK200!

Software

I will be exploring the software of the DK200 in a follow up post. Stay tuned!

The real meaning of 10^9

I recently bought a 128GB USB mass storage device from Amazon here in Germany. The price I paid for this unbelievable amount of storage in my pocket? Just 22 Euros.

As you maybe know from a previous article, I’m never exactly pleased when I buy a storage device and find out that the manufacturer is actually selling less capacity than they’re advertising. Rather than turn this into another rant, I’d like to start a discussion with storage manufacturers about what 10^9 means.

First, some definitions:

Gigabyte: 1,000,000,000 bytes is the actual number of bytes in a “gigabyte” which has historically had the acronym “GB” shown.

And when you think about it, it makes sense. Recall your SI units, where you have kilo (1,000), mega (1,000,000), giga (1,000,000,000), tera (1,000,000,000,000), peta (1,000,000,000,000,000)… you get the point.

Gibibyte: 1,073,741,824 bytes. This is the number of bytes in “GiB” which is what computers typically operate in, because it’s a power of 2 (2^30).

So, why the confusion? Well, when you buy a computer with 8 “GB” of RAM, you’re actually buying a computer with 8 GiB of RAM. But, for historical reasons, it’s much more common to see electronics advertised with “GB” instead of “GiB” (although in recent years things seem to be changing, at least on the software side).

Some smart executive at a storage company long ago figured out that if they were advertising products as having 1 GB of capacity, that was actually 10^9 bytes, not 2^30 bytes, and they could increase profits if they started selling devices which were only 1,000,000,000 bytes instead of 1,073,741,824 bytes. That’s like 7% less storage!

https://commons.wikimedia.org/wiki/File:High_five!!.jpg

The celebration probably looked something like this

And in fact, storage manufacturers have been quite clear about this for a long time. They state quite clearly, in small print, on the back of the box or at the bottom of their product webpage, that the actual size of a Gigabyte is really 1,000,000,000 bytes. So, when you plug that shiny new storage device into your computer, and see 119.2GiB, well that’s just you failing to do the math to account for the difference between 10^9 and 2^30.

In fact, many manufacturers have support pages dedicated to outraged people who buy a device and haven’t read the fine print. Here is the SanDisk website explaining device capacity:
http://kb.sandisk.com/app/answers/detail/a_id/46/kw/capacity

I am not here trying to argue that electronics manufacturers should advertise storage devices advertised in GiB. They’re correctly advertising the capacity of the devices in Gigabytes, the SI unit. It would probably help SanDisk even more if on their website they used the appropriate definition for 2^30, which is Gibibyte. But as I said, consumers have gotten used to reading “GB” so when they see “GiB” they don’t understand what the difference is.

I know, I can hear you thinking right now “Okay, so what? Get to the point already, you’ve been droning on for over 400 words. Hey, are you paid by the word?”

First, I make no money from this website. And second, I said this was a discussion, not a rant. In a discussion, you must provide context and frame the problem. Otherwise it’s just blatent complaining.

Remember that SanDisk page (pictured above) where they helpfully explained the difference between a Gigabyte and Gibibyte for us? Well, I didn’t show you the whole page. Here is what is written directly after the screenshot pictured above explaining a gibibyte:
sandisk_capacity_explanation_revealed

So, basically what SanDisk is telling us here is that the actual size of the device isn’t even the advertised capacity multiplied by 10^9 bytes, it’s actually less. This is basically the legal equivalent of them saying “trust us, it really has 128,000,000,000 bytes inside, but you can’t use them all.”

Somehow, our governments have decided that this kind of advertising is legal.

I went back and looked at the Amazon.de page where I bought the product. It might shock you, dear reader, but there was no fine bullet point in the specifications saying “actual user storage less”:
amazon_de_128GB_cruzer_blade

This is understandable. If people saw “actual user storage less” mentioned in the product advertisement, they would probably be suspicious of the amount of storage they were actually buying, and sales would suffer.

I thought I would go look at other retailers to see if “actual user storage less” was mentioned anywhere on their websites. Here is the same product listed on Amazon.com:
amazon_com_128GB_cruzer_blade

What about NewEgg.com?
newegg_com_128GB_cruzer_blade

Okay, so the companies selling these devices aren’t overly eager to include this fine print, which SanDisk actually includes on their website. In small text, at the bottom of the page:
sandisk_com_128GB_cruzer_blade

I emailed SanDisk about this to ask why the “actual user storage less” wasn’t mentioned on any retailers website, and they responded:
sandisk_reply_1

While I disagree with their reply, I understand that retailers have some freedoms in how they advertise a product. However, with this in mind, I am sure that a small army of SanDisk lawyers would co-sign a cease and desist letter if I started advertising their products in any way which they determined was harming their brand value. But then something curious happened…

sandisk_reply_2

SanDisk claims that they don’t have any control over how retailers advertise their product, but then they state that these companies are “authorised distributors and resellers.”

Given the incredibly high percentage of counterfeit products being sold these days under the label of a well known brand, it’s clear that manufacturers need a trustworthy outlet to sell their goods, or consumers might begin to doubt the quality of their brand. That’s the economic impact of “electronics priacy” [PDF].

Managing your supply chain and maintaining your brand image costs companies millions, if not billions, of dollars every year. It’s serious business. People go to jail for importing and selling counterfeit products.

So, when a company claims that they have no control over how their product is advertised, I find that a bit difficult to believe. Legally they may not have an obligation to require retailers mention “actual user storage less” but morally and ethically they should ensure that their retailers do not advertise their product in a misleading way.

SanDisk is selling a “128GB” USB stick, which has a raw capacity of ~125GB (116GiB):
sandisk_cruzer_128GB_fdisk

They also mention in small text, on the back of the package, that “actual user storage less.” Too bad they didn’t state this anywhere on the actual retail page.

At this point, anyone who is sane will do the math and ask the question “You received 2.2% less capacity than was advertised. Why have you wasted your time writing about this?” and that’s because I’m scared of the precedent this is setting.

It’s true, it’s seemingly pointless to sit here and discuss the missing 2.2%. But 10 years ago, you would spend a lot of money to buy a USB stick which was 2GB. Even now, most people pay phone companies tens of dollars per month to transfer 3GB over 3G or LTE, or $90 per month if you’re unlucky enough to own a smartphone in Canada:
canadian_cell_plans

So even though as a percentage, it’s relatively small, it a not-insignificant amount of capacity that’s missing. If this was a 1TB device, you would be missing 23GB, and this is even before we get into Giga versus Gibi and formatting…

When I buy a device where the primary function is storage, I expect to be buying a device which can contain $CAPACITY * 10^9 bytes of data. This is important for things like data recovery. If I need to make an exact duplicate of the data on a storage device which is 128GB, I don’t want to have to worry about buying a device from a specific manufacturer, model, or serial number range to be able to store the data. This isn’t swapping the PCB on a broken hard drive, it’s just buying a simple storage device!

I’m scared of what message we’re sending to manufacturers when we allow them to sell us products with vague statements like “actual user storage less” and what it means for the future of the industry.

If I buy a smartphone which has 16GB of space, I expect to be able to use less, because the primary function of a smartphone is to be a pocket computer. I understand that capacity is required for the operating system, and that the actual capacity available to me will be less than the advertised amount.

But to buy a device whose sole purpose is to store data, and have that device provide less capacity than advertised. That’s scary, not for the 2.2% that I can’t use today, but because tomorrow it might be 5%, and in 10 years 20%.

Why is the capacity less? Perhaps they’re using NAND which doesn’t have space over provisioned for error correction and wear levelling, allowing them to fit a few more on a wafer. Or maybe the chips have bad regions which they’ve mapped around, meaning you get a slightly smaller capacity. I doubt we’ll ever get an explanation apart from “actual user storage less.”

SanDisk isn’t the only manufacturer doing this, but they’re certainly the worst offender that I’ve found. Recently I purchased some 16GB micro SDHC cards from Transcend, and they’re 0.5% under capacity as well. Luckily I have found that Samsung’s EVO line appears to at least provide the advertised capacity in 10^9 bytes. But, how much longer until everyone clues in to the get out of jail free card that is “actual user storage less” and starts selling devices under the 10^9 capacity?

The moral of the story here is: complaining on the internet is useless. Vote with your wallet. Return anything which is not actually 10^9 bytes of capacity, and rate it accordingly to warn other users. I’m not going to support a brand which, in my opinion, allows retailers to advertise their products in misleading ways. Besides, the performance sucked (4.5MB/s sequential write). You get what you pay for.

Linux 4.5 on a Bay Trail tablet

This post is a short update to my original article on booting Arch Linux on a Bay Trail tablet.

I originally wrote this for 4.4.5, but I wasn’t fast enough, and 4.5 was released before the post was completed, so might as well continue with a 4.5 kernel.

To simplify the build process I took the PKGBUILD for linux-mainline in AUR and modified it to build a mainline kernel with patches for SDIO WiFi on BayTrail.

If you’d like to build the kernel yourself (and you happen to run Arch Linux) you can download the PKGBUILD.

The firmware for the rtl8723bs card is in its own package, in keeping with the Arch Linux best practices for separating firmware from the kernel package. Download the firmware PKGBUILD.

Or, if you’d rather just have a newer kernel on your tablet which is already running Arch Linux, you can download the pre-built kernel package, and the firmware package.

I will be submitting both of these packages to AUR shortly.

Turns out you can actually get GRUB working with a menu if you build a standalone version of grub. However, the issue is that even though the grub menu works, there’s some issue with modesetting and you’ll never see any console after grub hands off to the kernel. You can download the standalone version of grub if you want to try, I wasn’t able to get any usable installer environment out of it. You can download standalone grub for ia32 (i686), you will also need grub.cfg.

$ tar -Jxf bootia32.tar.xz
$ cp bootia32.efi /mnt/archiso/EFI/boot/bootia32.efi
$ cp grub.cfg /mnt/archiso/EFI/boot/grub.cfg

Since grub video handoff isn’t working well, the only way I was able to successfully boot was to drop to command line by pressing c at the menu, and typing the following:

set root=hd0,msdos1
linux /arch/boot/x86_64/vmlinuz archisobasedir=arch archisolabel=ARCH_201603 video=VGA-1:800x1280@75e
initrd /arch/boot/x86_64/archiso.img
boot

There is a small issue with kernel oops, which has been present since at least 4.4.5:

[  164.281827] NMI watchdog: Watchdog detected hard LOCKUP on cpu 2
[  164.281913] Modules linked in:
[  164.281962]  intel_rapl intel_soc_dts_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw iTCO_wdt snd_soc_sst_bytcr_rt5640 iTCO_vendor_support hid_multitouch gf128mul glue_helper dcdbas ablk_helper cryptd pcspkr hci_uart snd_intel_sst_acpi mei_txe joydev input_leds snd_intel_sst_core btbcm snd_soc_rt5640 evdev snd_soc_sst_mfld_platform mousedev btintel mei lpc_ich mac_hid snd_soc_rl6231 thermal snd_soc_sst_match dw_dmac dw_dmac_core tpm_crb snd_soc_core bluetooth processor_thermal_device int3400_thermal int3403_thermal acpi_thermal_rel int3402_thermal i2c_hid int340x_thermal_zone snd_compress intel_soc_dts_iosf tpm_tis rfkill_gpio snd_pcm_dmaengine battery ac97_bus ac spi_pxa2xx_platform crc16 tpm snd_pcm i2c_designware_platform
[  164.283185]  acpi_pad i2c_designware_core 8250_dw snd_timer snd processor soundcore sch_fq_codel nfs lockd grace sunrpc fscache ip_tables x_tables overlay squashfs loop nls_iso8859_1 nls_cp437 vfat fat sd_mod uas usb_storage scsi_mod hid_generic usbhid hid i915 mmc_block button i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crc32c_intel xhci_pci drm xhci_hcd intel_gtt wmi serio video sdhci_acpi sdhci led_class r8723bs(O) cfg80211 rfkill mmc_core
[  164.283978] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G           O    4.5.0-byt #1
[  164.284073] Hardware name: Dell Inc. Venue 8 Pro 3845/XXXXXX, BIOS A02 12/29/2014
[  164.284169]  0000000000000086 9ad5a4512f59852f ffff880039d05b50 ffffffff812d25d1
[  164.284284]  0000000000000000 0000000000000000 ffff880039d05b68 ffffffff81116550
[  164.284399]  ffff880038ca8000 ffff880039d05ba0 ffffffff81156b4c 0000000000000001
[  164.284513] Call Trace:
[  164.284552]    [] dump_stack+0x63/0x82
[  164.284645]  [] watchdog_overflow_callback+0xe0/0xf0
[  164.284733]  [] __perf_event_overflow+0x8c/0x1d0
[  164.284815]  [] perf_event_overflow+0x14/0x20
[  164.284894]  [] intel_pmu_handle_irq+0x1e1/0x460
[  164.284980]  [] perf_event_nmi_handler+0x28/0x50
[  164.285062]  [] nmi_handle+0x5e/0x130
[  164.285133]  [] default_do_nmi+0x48/0x120
[  164.285207]  [] do_nmi+0xe2/0x130
[  164.285274]  [] end_repeat_nmi+0x1a/0x1e
[  164.285349]  [] ? poll_idle+0x39/0x80
[  164.285420]  [] ? poll_idle+0x39/0x80
[  164.285490]  [] ? poll_idle+0x39/0x80
[  164.285558]  <>  [] cpuidle_enter_state+0xf3/0x2f0
[  164.285655]  [] cpuidle_enter+0x17/0x20
[  164.285728]  [] call_cpuidle+0x2a/0x40
[  164.285800]  [] cpu_startup_entry+0x2c5/0x3a0
[  164.285878]  [] start_secondary+0x165/0x1a0
[  164.285964] INFO: NMI handler (perf_event_nmi_handler) took too long to run: 4.144 msecs
[  164.286069] perf interrupt took too long (32852 > 2495), lowering kernel.perf_event_max_sample_rate to 50100
[  172.707012] perf interrupt took too long (32619 > 4960), lowering kernel.perf_event_max_sample_rate to 25200

You’ll see a lot of these, however WiFi still continues to work, and the tablet didn’t kernel panic for me in the installer environment.

Hopefully someone finds this useful. I’ll have a write up on installing and using Arch Linux on the tablet in the coming weeks.