Today I am going to discuss how to build and package your own software for OpenWRT.
When I say “your own software” in this case I am referring to a C program which you want to cross-compile for the target SoC and install using the opkg package manager included in OpenWRT.
The program I wrote is a little more complicated than your standard “Hello World” application. Here’s what I wanted to do:
1) use libconfig to read a configuration file in /etc/config/ and then perform actions based on the configuration described in this file
2) use sqlite3 to create a database
3) write some meaningful data to the database
Here’s the program flow:
1) Open /etc/config/example-sqlite and read the values into variables
2) Open (or create) a new SQLite3 database file at the location defined in the above configuration file
3) Determine if the SQLite file is initialized with the target table we want to write to, and if not, create the table
4) Write the system load average to the database
5) Quit
To recap, this program is different from “Hello World” in the following ways:
1) It must read and understand a configuration file in libconfig syntax; this requires linking against the libconfig library, which we must tell opkg is a dependency
2) It must create or open an SQLite 3 database; this requires linking against the sqlite3 library, which we must tell opkg is a depenedency
3) It must perform some useful operations on this SQLite file
Let’s start with compiling the C file on your native architecture. Sure, you can just use cc/gcc from bash, but this isn’t any good to OpenWRT SDK, which expects that each package will have a makefile which can be used to compile the software.
load2sqlite.c
#include <sys/types.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <sqlite3.h>
#include <libconfig.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
int main(int argc, char *argv[]) {
// ...
Most importantly above, we are including sqlite3.h for SQLite support, libconfig.h, and sys/stat.h, fcntl.h,errno.h to check if the SQLite3 database file exists or not.
You can compile this by hand quite easily, just by doing:
cc load2sqlite.c -lsqlite3 -lconfig -o load2sqlite
Okay, but how do we make this ready for OpenWRT SDK? By writing a makefile!
makefile
PROFILE = -O2 -s CFLAGS = $(PROFILE) LDFLAGS = -lsqlite3 -lconfig all: main # build it main: $(CC) $(CFLAGS) load2sqlite.c $(LDFLAGS) -o load2sqlite # clean it clean: rm load2sqlite
Okay, so now if you type make in the directory, magically you will end up with an executable called load2sqlite!
But, this is a native binary, and it’s somewhat unlikely that your OpenWRT device is on the same architecture.
load2sqlite: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=9661b88e92b553d0556cbeeafccf04d2526c770f, stripped
If you run it, you’ll see that it looks for the sqlite database file, can’t find it, and so initalizes a new one with the “readings” table.
[hmartin@localhost src]$ ./load2sqlite Database file /tmp/sqlite3.db does not exist Initialized database with readings table [hmartin@localhost src]$ echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 2015-10-28 22:48:42|0.57|0.56|0.57
And if you run it again, without removing the SQLite3 file that was created, you’ll see this output:
[hmartin@localhost src]$ ./load2sqlite SQLite database opened Found readings table [hmartin@localhost src]$ echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 2015-10-28 22:48:42|0.57|0.56|0.57 2015-10-28 22:49:00|0.47|0.54|0.57
Before we proceed further, I want to show you the directory structure so you have an idea of where we just were when we did this compilation. We are currently in the the src directory.
load2sqlite/
|-- Makefile
|-- README
`-- src
|-- load2sqlite.c
|-- load2sqlite.conf
`-- makefile
Now let’s move up to the load2sqlite directory and work on the OpenWRT Makefile (seen above).
Here is the complete file, and then we will discuss it section by section:
Makefile
# # Copyright (C) 2006-2015 OpenWrt.org # # This is free software, licensed under the GNU General Public License v2. # See /LICENSE for more information. # include $(TOPDIR)/rules.mk PKG_NAME:=load2sqlite PKG_VERSION:=1.0.1 PKG_RELEASE:=5 PKG_MAINTAINER:=Hal Martin PKG_LICENSE:=GPL-2 PKG_CONFIG_DEPENDS:=libsqlite3 libconfig include $(INCLUDE_DIR)/package.mk PKG_BUILD_DIR := $(BUILD_DIR)/$(PKG_NAME)-$(PKG_VERSION) TARGET_LDFLAGS+= \ -Wl,-rpath-link=$(STAGING_DIR)/usr/lib \ -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/libconfig/lib \ -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/sqlite/lib define Package/load2sqlite SECTION:=utils CATEGORY:=Utilities DEPENDS:=+libsqlite3 +libconfig TITLE:=SQLite example program, creates or opens a user defined SQLite database URL:=https://github.com/halmartin/load2sqlite MENU:=1 endef define Package/load2sqlite/description Example SQLite is a sample program built using libsqlite3 and libconfig which creates or opens a user-defined SQLite3 database and performs some simple verification checks on the file to ensure that the target table (readings) exists, and if not creates the table, then inserts a row with the current system time, and the load (1 minute, 5 minute, 15 minute). endef define Build/Prepare mkdir -p $(PKG_BUILD_DIR) $(CP) ./src/* $(PKG_BUILD_DIR)/ endef define Build/Configure endef define Build/Compile $(MAKE) -C $(PKG_BUILD_DIR) $(TARGET_CONFIGURE_OPTS) endef define Package/load2sqlite/install $(INSTALL_DIR) $(1)/bin $(INSTALL_BIN) $(PKG_BUILD_DIR)/load2sqlite $(1)/bin/ $(INSTALL_DIR) $(1)/etc/config $(INSTALL_CONF) $(PKG_BUILD_DIR)/load2sqlite.conf $(1)/etc/config/load2sqlite endef $(eval $(call BuildPackage,load2sqlite))
If you clone the OpenWRT source and take a look at basically any package, you’ll see a Makefile that looks similar to the one above.
Let’s look at the package information:
PKG_NAME:=load2sqlite PKG_VERSION:=1.0.1 PKG_RELEASE:=5 PKG_MAINTAINER:=Hal Martin PKG_LICENSE:=GPL-2
Here is where we define core details of our package, such as the name (e.g. what opkg will know it as), the version (useful for upgrading later), maintainer, and license.
TARGET_LDFLAGS+= \ -Wl,-rpath-link=$(STAGING_DIR)/usr/lib \ -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/libconfig/lib \ -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/sqlite/lib
Since we want to build a program which links against external libraries, we must also tell the compiler where to find the header files for these libraries, so that the linking process does not fail during compilation. Above you can see that we are linking to libconfig and sqlite libraries.
define Package/load2sqlite SECTION:=utils CATEGORY:=Utilities DEPENDS:=+libsqlite3 +libconfig TITLE:=SQLite example program, creates or opens a user defined SQLite database URL:=https://github.com/halmartin/load2sqlite MENU:=1 endef
This is where you define the package for the OpenWRT build system and declare things like dependencies, and the description that will be present when you run menuconfig (which is how you will select your package to be built as part of an image).
Without declaring dependencies, you may find that you can build, package, and install your software, but it won’t run! So, by declaring the dependencies (packages which provide the libraries we link against) we ensure that when we type opkg install load2sqlite and libconfig and libsqlite3 are not installed, opkg knows to go and install them before installing our program. Now we can safely run the program because all the required libraries are installed on the device!
define Build/Prepare mkdir -p $(PKG_BUILD_DIR) $(CP) ./src/* $(PKG_BUILD_DIR)/ endef define Build/Configure endef
Since our utility is quite simple, as *NIX software goes, the preparation steps are to create the build directory and copy the source from the source directory to the build directory. Since there is nothing to configure in our sample program, the configure step is empty (otherwise the OpenWRT build system will attempt to configure the package and fail because we haven’t bothered to implement this).
define Build/Compile $(MAKE) -C $(PKG_BUILD_DIR) $(TARGET_CONFIGURE_OPTS) endef define Package/load2sqlite/install $(INSTALL_DIR) $(1)/bin $(INSTALL_BIN) $(PKG_BUILD_DIR)/load2sqlite $(1)/bin/ $(INSTALL_DIR) $(1)/etc/config $(INSTALL_CONF) $(PKG_BUILD_DIR)/load2sqlite.conf $(1)/etc/config/load2sqlite endef
Finally, compile and install the software. As you can see above, I didn’t include an install directive in the makefile of the application, it is instead done manually within the OpenWRT Makefile. This is your choice, since I was designing this program specifically to run on OpenWRT, I saw no need to incorporate the installation steps in the makefile of the program.
And, finally:
$(eval $(call BuildPackage,load2sqlite))
This line is required for OpenWRT to build the package. Forget this line, and you will sit there wondering why your package is not being built!
Okay, now we have prepared our software to be built for OpenWRT. It would be stupid of me to get this far and not tell you how to compile it using the OpenWRT toolchain!
Following the excellent OpenWRT documentation, we need to set up a buildroot.
Install the dependencies (instructions for Debian/Ubuntu):
sudo apt-get install git-core build-essential libssl-dev libncurses5-dev unzip subversion mercurial
Clone the OpenWRT Chaos Calmer release:
git clone git://git.openwrt.org/15.05/openwrt.git
I find that the stock OpenWRT repository is a bit light on some of the software I like to have on my routers, so I take step 3 and install the additional feeds:
cd openwrt ./scripts/feeds update -a ./scripts/feeds install -a
Follow step 4 to ensure you have all the required dependencies installed on your host system!
make defconfig make prereq # don't forget to copy load2sqlite to package/utils/ before running this step, or the package won't appear in the menu! make menuconfig
If everything has gone well thus far (e.g. no errors in the OpenWRT Makefile, and you put load2sqlite in package/utils/ then you should see the following in your menuconfig:
Now I already have an official OpenWRT build installed on my router, so I don’t need to build an entire image, just the package I want to install. To do this, we must first build the cross compilation toolchain required to compile for a different CPU architecture.
Warning: the OpenWRT buildroot is around 6GB on disk, so ensure you have the necessary space before starting!
make tools/install # this will take a while the first time make toolchain/install # this will also take a while the first time
When we have the tools and toolchain compiled, we can compile our package:
make package/load2sqlite/compile
This will create an ipkg file in bin/ramips/packages/base/load2sqlite_1.0.1-5_ramips_24kec.ipk which we need to copy to our router to install:
scp bin/ramips/packages/base/load2sqlite_1.0.1-5_ramips_24kec.ipk [email protected]:/tmp/ # scp completes ssh [email protected] [email protected]'s password: BusyBox v1.23.2 (2015-07-25 03:03:02 CEST) built-in shell (ash) _______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M ----------------------------------------------------- CHAOS CALMER (15.05, r46767) ----------------------------------------------------- * 1 1/2 oz Gin Shake with a glassful * 1/4 oz Triple Sec of broken ice and pour * 3/4 oz Lime Juice unstrained into a goblet. * 1 1/2 oz Orange Juice * 1 tsp. Grenadine Syrup ----------------------------------------------------- root@OpenWrt:~# opkg install /tmp/load2sqlite_1.0.1-5_ramips_24kec.ipk Installing load2sqlite (1.0.1-4) to root... Installing libsqlite3 (3081101-1) to root... Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/packages/libsqlite3_3081101-1_ramips_24kec.ipk. Installing libpthread (0.9.33.2-1) to root... Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/base/libpthread_0.9.33.2-1_ramips_24kec.ipk. Installing libconfig (1.4.9-1) to root... Downloading http://downloads.openwrt.org/chaos_calmer/15.05/ramips/mt7620/packages/base/libconfig_1.4.9-1_ramips_24kec.ipk. Configuring libpthread. Configuring libconfig. Configuring libsqlite3. Configuring load2sqlite.
Now that our package is installed, we can test it!
root@OpenWrt:~# /bin/load2sqlite Database file /tmp/sqlite3.db does not exist Initialized database with readings table
If you install sqlite3-cli we can inspect the row added to the file:
root@OpenWrt:~# opkg install sqlite3-cli root@OpenWrt:~# echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 2015-10-31 20:47:33|0.76|0.4|0.25
Since this is just an example program, it is one-shot (e.g. not a daemon). If you really do want to track the load of our OpenWRT router, just add /bin/load2sqlite to crontab (e.g. every hour) and you’ll have this tracking info in the SQLite database.
If you run it multiple times, you get another row added to the file each time the program is run:
root@OpenWrt:~# /bin/load2sqlite SQLite database opened Found readings table root@OpenWrt:~# echo "select * from readings;" | sqlite3 /tmp/sqlite3.db 2015-10-31 20:47:33|0.76|0.4|0.25 2015-10-31 20:53:43|0.02|0.2|0.22 2015-10-31 21:23:19|0.08|0.04|0.05
Note that by default the file is saved to /tmp/, which on OpenWRT is a ramdisk. This means that the file will be lost when you reboot, or if you leave it running unattended for too long, the file size will grow to the point where the ramdisk will consume all available memory and the router will crash. For this reason, I suggest you modify the configuration file /etc/config/load2sqlite to point to non-volatile storage (such as a USB stick).
Source code: https://github.com/halmartin/load2sqlite
Why write another OpenWRT software guide?
Well, while I was looking for resources on how to build and package software for OpenWRT, I ran into a lot of posts about people compiling simple “Hello World” programs for OpenWRT, but for my particular use case, I wanted to utilize multiple libraries in my program, and I couldn’t find any good instructions on how to compile a program with linked libraries for OpenWRT.
Disclaimer: I’m not a C expert, so maybe there are some headers there which are not strictly necessary, but it works for me and the executable size is quite small.
If you wish to further reduce the size of your executable, you can tell the compiler to strip it of the symbol table and relocation information. Do this by appending -s to the PROFILE line in the makefile. When I did this on my laptop, the output went from 9.9KB to 7.0KB, or a savings of 30%
I have tested this on Chaos Calmer (15.05), and I expect the instructions would also work on Barrier Breaker (14.07) however I didn’t try this, so I cannot say certainly that it will work.