Tag Archives: iot

IoT sensors and time series databases

In this article we are going to look at a few uses for low-cost sensors and how they can be combined with a time series database (TSDB) and a web front-end to easily visualize the data. For privacy reasons, I will describe a standalone use case where you have the time series database running on low-cost hardware such as a Raspberry Pi (or Chinese equivalent), so the data never leaves your house and the IoT sensors are not directly exposed to the internet.

Energy monitoring

The Sonoff POW is a $12 wireless relay that includes a power measurement IC capable of measuring energy consumption, voltage, current, etc.

The Sonoff POW is based on the ESP8266, and there are a number of third-party firmwares available which add additional functionality like support for MQTT, InfluxDB, Domoticz, Amazon Alexa, etc. The most popular third-party seems to be ESPurna, which is what I’m using.

ESPurna is not infallible, and does occasionally crash. When that happens, the relay cycles, disrupting power to whatever is connected. Since I’m monitoring things I don’t want to be randomly power cycled, I soldered across the relay to prevent it from shutting off the loads. This turns the Sonoff POW from a wireless relay with energy monitoring to simply an energy monitor. However for my purpose that’s fine.

Bypassing the relay in the Sonoff POW requires opening the case and soldering across the relay. Working on the Sonoff POW should only be done when it is not connected to AC (mains power)! After soldering, you should confirm with a multimeter that you have correctly bypassed the relay and not created a short circuit. There have been several iterations of the Sonoff POW PCB, so I cannot provide universal instructions on how to bypass the relay.

On the latest Sonoff POW hardware I own (purchased in mid-2017), you can bypass the relay by soldering a wire (shown in red) between the relay input and output:

The small gold coloured object is the shunt resistor used to measure the current consumed by the load. To keep the energy monitoring functionality intact, it is important that you only solder after the shunt resistor (to the left), not before (to the right), otherwise the shunt resistor will not be in series with the load and the measured current will be 0.

Environmental monitoring

The Wemos D1 mini is a “mini WiFi board” with a large number of “shields” incorporating various sensors or other expansion options.

I was drawn to the Wemos D1 mini because it is supported by MicroPython as well as ESPurna (though not for my intended use case). Since there are many shields available, you can just stack modules to get the desired functionality instead of messing around on a breadboard or soldering onto protoboard.

The Wemos D1 mini is also cheap, you can buy it from China for under $3 with free shipping (at least to the EU). The modules are also quite inexpensive when ordered from China, as long as you don’t mind waiting 4-6 weeks for delivery.

Since ESPurna only supports the Wemos D1 mini with the relay shield, and I wanted to do temperature/humidity/pressure monitoring, I decided to use MicroPython since it has the lowest barrier to entry. Flashing MicroPython on the Wemos D1 mini wasn’t too complicated, there is a forum thread describing how to flash it.

I created a simple python script to report the temperature and humidity to the InfluxDB server every minute. Overall it works well, the only issue I’ve run into is that there is no watchdog on the ESP8266, so if the urequests.post() fails for some reason (DNS resolution issue, packet loss, alignment of the stars) you have to manually reset the sensor using the reset button on the side.

Since these are just around my apartment, I added a “meatware monitoring” feature. When the POST is in progress, the LED on the Wemos is enabled. For a normal POST, the LED will just blink for around a second. If I walk past a sensor and notice the LED is on solid, I just press the reset button. This is not very “production ready” but I’m only monitoring the temperature and humidity for fun, so the motivation to resolve this bug is not very high. I will accept any pull requests to improve the functionality.

Time series database (TSDB)

Time series databases are a relatively new and hyped type of database, as you can probably gather from how incomplete the Wikipedia page is compared to relational databases.

For my application of 5 sensors reporting values every minute or so, there’s no reason a relational database like PostgreSQL couldn’t be used instead. But it’s helpful to learn a new technology, and InfluxDB offers some benefits over a relational database:

  • Engineered for time series data
  • collectd API

These are only scratching the surface of InfluxDB’s features, but the HTTP and collectd APIs reduced the amount of effort needed for this project. Otherwise I would have had to write an HTTP API to accept readings and insert them into a relational database. collectd is also useful to collect performance metrics from devices running Linux or BSD, but that’s beyond the scope of what I want to discuss today.

For the Sonoff running ESPurna, there’s no additional programming required as InfluxDB is supported by default. Simply enter the URL of your InfluxDB server’s HTTP API and wait for the sensors to report readings.

Visualising the data

Now, it’s great that we can send data to InfluxDB with very little effort via the HTTP API. We can of course run queries on the data from the influx cli, however this isn’t very useful for getting a quick impression of the data.

time value
---- -----
1512599873138880467 27.23
1512599933609113738 26.57
1512599994180436248 26.5
1512600054652895777 26.31
1512600115149476539 26.31
1512600175695516312 26.26
1512600256017317051 26.21
1512600316488957374 26.07
1512600376985207309 25.99
1512600437407006181 26.02

To visualise the data, I’ve chosen to use Grafana. Grafana is free software that you can use to visualise data from a variety of data sources such as OpenTSDB, InfluxDB, graphite, elasticsearch, and more.

Coupling Grafana with the InfluxDB data source from the Sonoff and the Wemos, we can build clever dashboards to visualise the sensor data:

Sonoff POW monitoring a fridge and microwave, you can see where the microwave was running

D1 Mini with SHT30 shield monitoring temperature and humidity,
can you see when the window was opened?

Security considerations
I would like to add that for security reasons if you are using any IoT devices at home, I would strongly recommend you consider isolating the devices to a separate WiFi access point and subnet to prevent them from communicating with devices on your main network. ISP supplied routers with a “Guest WiFi” mode should be capable of implementing this. Alternatively you can find inexpensive routers such as the Nexx WT3020H which support OpenWrt/LEDE and could be used to implement this.

You could in theory implement this on an SBC with WiFi supporting AP mode (such as the Orange Pi Zero), negating the need for a separate WiFi AP. However you are either faced with a SBC with very limited resources (the Orange Pi Zero has only 512MB of RAM), or an SBC with higher price than a Raspberry Pi with a WiFi router such as the Nexx WT3020H.

Tying it all together
We’ve looked at sensors, InfluxDB, and Grafana in this article. I haven’t mentioned until now that I’m running all of this on an Orange Pi PC, a small single board computer based on an Allwinner processor. For my use case, this hardware is low-energy, low-cost, and meets the performance needs of InfluxDB and Grafana.

There is nothing preventing you from running all of the above software on a different architecture (e.g. Docker on an x86). I chose ARM purely because I had the hardware available, and it is low power. If you’re building a monitoring system from scratch and your processing needs are not significant, then a SBC like the Raspberry Pi or Orange Pi PC is a very inexpensive server you can use with sensors.

I want to close by leaving some installation instructions if you are interested in implementing this yourself. This article is mostly just to inspire you to do your own projects, and is not a novel application of sensors, databases, or data visualization. So in this case, I will leave some links to other people who have written detailed instructions on how to install and configure InfluxDB and Grafana on ARM.

Intel DK200 IoT Gateway

Earlier this year I was at a conference and heard from other attendees that the Intel booth was giving away IoT gateways. Never one to turn down free conference swag, I hurried over to the Intel booth and was told to pick up a gateway out of a pallet of boxes just delivered (and rapidly disappearing).

The Intel IoT gateway series is codenamed Moon Island, but the design targeting the transportation market is codenamed Clanton Hill. Clanton Hill known to us mortals as the “Intel DK200 Series Gateway Solution for the Internet of Things (IoT)” quite the mouthful.

Let’s get down to it.

Unless you happen to be at a conference where Intel reps are handing these out like candy, I don’t think it’s practical to try and buy one yourself:


Some interesting details to note about this product:

  1. Although released in 2014, the DK200 still costs more than the new MacBook (3,712.50 EUR versus 3,199 EUR)
  2. It’s End Of Life

When a low volume product goes EOL and you still have stock, I guess giving it away at conferences is the next logical step.

Hardware Specifications
The DK200 (datasheet) is targeted toward the transportation industry, and it really shows in the appearance of the device:

Only available in 'Cosmic Black'

Only available in ‘Cosmic Black’

I don’t work in the transportation industry, and have never seen connectors that look like this before. They’re very well made, and I suspect probably do a good job of keeping dust, dirt, and debris out of the ports. Since I don’t wish to make a mess by throwing dirt and debris at it, I’m going to have to trust the engineers who designed it.

The build quality is quite good, as one might expect from a device selling for 3,700 EUR. Nearly every screw is secured with loctite to prevent vibration from loosening them:

DK200 screw with loctite

No pentalobe nonsense here

However, I was surprised to find that despite all the physical hardening applied to the enclosure, I couldn’t find any information on an IP rating. In fact the top and bottom of the case don’t appear to offer any additional dust or water seal. There’s clearly been a lot of thought put into the design of this enclosure to withstand vibration and dirt, so it’s strange that there doesn’t seem to be water protection of any kind.

The Intel Quark series SoC was introduced in late 2013. The X1020D in the DK200 is a single core SoC based around a 80486 core running at 400MHz, with modern I/O and memory.


In 2014 a leaked product roadmap suggested a successor to the X1000 series named “Dublin Bay” to be released in 2015. Then news emerged that “Dublin Bay” had been cancelled, to be replaced by “Liffy Island” and “Seal Beach” which would be released in 2015. As of late 2016 Intel has not released a direct successor to the X1000 series, and there is no new news of “Liffy Island” or “Seal Beach” being cancelled (or released). So it’s anyone’s guess whether Intel is still even interested in the IoT gateway market.

The DK200 doesn’t include any of the typical storage buses like SATA, NVMe, or NAND (EMMC). This is not overly surprising given the embedded nature of the hardware (requiring lower power) and the simplicity of the Quark processor.

The only storage option is a micro SDHC card, and the DK200 includes an 8GB class 4 micro SD card:

Given that it’s a class 4 card, the performance is quite poor. Use of an SD card for storage isn’t a bad decision per se, but the DK200 uses ext3 for the root partition. Ext3 is not a flash aware filesystem. SD cards have only basic wear leveling, and ext3 has no wear leveling. So it hardly seems like the appropriate combination of storage and filesystem for a headless embedded device with an expected lifetime of 5-10 years.

Input and output

  • Dual 100Mbit Ethernet controllers
  • 3 x USB 2.0 host and 1x device
  • Audio in/out
  • CAN bus
  • RS-232
  • GPIO
  • 1x half-height mini PCI-e slot (populated with Intel 7260)
  • 1x full-height mini PCI-e slot (unpopulated; for 3G modem/GPS)

The Intel documentation also mentions ZigBee, however this is an external device, presumably attached via the USB bus.

Power consumption
Development platforms aren’t known for being highly optimised devices. They often include extra I/O which would not necesssarily be included in the final product, and as such do not have the same energy efficiency as a finished product.

This being said, I was quite surprised that a device intended for 24/7 operation in an embedded environment, and especially serving the “Internet of Things” market, could be so energy inefficient. Issuing a poweroff command in Linux results in the platform going into an S5 (shutdown) state. I was surprised to discover that the energy consumption in the S5 state is 2W. This seems quite high for a device which includes an ignition input for automatic power-on and shutdown.

When booting, the device peaks at 7.9W consumption, while the idle power consumption is 7.5W. This is almost certainly due to the added peripherals as the TDP of the Quark processor is only 2W.

It’s difficult to see how Intel expects the Quark platform to compete with ARM. My PandaBoard ES, an ARM-based development board from 2011, peaks at 4W, idles at 2W, and draws nothing when off. Now some might argue that comparing an ARM board from 2011 with an Intel IoT gateway from 2014 isn’t valid, but they do have a lot of similar features. Now, I will grant that the PandaBoard is not in a rugged enclosure with fancy connectors, but since it cost 95% less than the DK200 does, there’s some room in the budget for an enclosure and funky connectors. And, since Texas Instruments has stopped making OMAP chips, the PandaBoard gets about the same amount of vendor support as the DK200!


I will be exploring the software of the DK200 in a follow up post. Stay tuned!