OTA Connect Developer Guide

Build a QEMU image

HERE OTA Connect lets you easily manage OTA updates to embedded devices running custom-built Yocto images. This is a guide for building a simple Yocto image that you can run in QEMU. This is a good way to get started if you don’t know yet what hardware your project will use, or if you just want to try out the features of HERE OTA Connect without worrying about physical devices.

Prerequisites

You’ll need a build machine with the following:

  • A x86-64 Linux distro supported by the Yocto project with the required packages installed.

    • On a Debian-based system, you should be able to install all the required packages with the following command:

      sudo apt install gawk wget git diffstat unzip texinfo gcc-multilib build-essential chrpath socat cpio python python3 python3-pip python3-pexpect python-dev xz-utils debianutils iputils-ping cpu-checker default-jre parted
    • Many/most distros that aren’t on the officially supported list will still work just fine—​feel free to give it a try with whatever you’re running.

  • 100GB+ of free disk space

  • 6GB+ of RAM

  • QEMU—​we recommend installing it from your distro’s package manager, e.g. sudo apt install qemu

  • repo

It’s possible use a virtual machine running Linux as your build machine. However, we don’t recommend it. It will be slower, and you’re more likely to run into difficult-to-troubleshoot issues. If you do want to use a VM despite this warning, though, make sure the VM has enough resources allocated to it. Along with the disk space and memory requirements above, we suggest allocating at least 4-6 CPU cores to the VM to speed up building.

Also, make sure that you’ve generated your provisioning credentials first.

Create your Yocto build environment

First, clone a manifest file for the quickstart project:

mkdir myproject
cd myproject
repo init -u https://github.com/advancedtelematic/updater-repo.git -m {yocto-branch}.xml
repo sync

This downloads the basic Yocto layers you need.

What is this actually doing?

Yocto is a set of tools, templates and methods for building Linux systems from scratch. Most Yocto-built systems use a common set of base layers. The Yocto project maintains a reference distribution called Poky; we include that as a base layer, then add layers containing hardware support for specific boards (in this case the QEMU VM). Finally, we include the meta-updater layer, which contains the platform-independent software the device and build system need to work with OTA Connect, and meta-updater-qemux86-64 for the device-specific code—​mostly the specialized bootloader code.

All of these layers are assembled into a built Linux system by Bitbake, the build tool of the Yocto Project, based on the instructions in the recipes inside the layers.

Now you can run the following script to get the environment set up:

source meta-updater/scripts/envsetup.sh qemux86-64

Customize your build

The environment setup script will have created a build directory and placed you in it. It also generates a configuration file, located at conf/local.conf. This file is where we’ll make our modifications to the base config.

To connect with your HERE OTA Connect account, you’ll need the provisioning credentials bundle you downloaded earlier. Add the following line to your local.conf to supply those credentials to the build:

SOTA_PACKED_CREDENTIALS = "/path/to/your/credentials.zip"

Optional configuration keys

  • Set image name

When you build a filesystem image, it gets automatically uploaded to OTA Connect. By default, the image will be named qemux86-64-ota, and you’ll see the various versions of the image under that name. You can also choose to set your own name as follows:

OSTREE_BRANCHNAME = "my-super-great-project"
  • Persistent Yocto shared state cache and download directory

Yocto caches its build artefacts to speed up future builds. By default, these are stored under the build directory of the current project. However, if you’re planning to build several different projects that have some shared base files, you might want them to share their cache directories, both to save space and speed up your builds. You can do that as follows:

SSTATE_DIR = "/path/to/your/shared-sstate"
DL_DIR = "/path/to/your/shared-download"
  • Add extra packages

There are quite a lot of packages available to install that aren’t installed by default. You can add extra packages to your image with IMAGE_INSTALL_append; for example, this will install vim:

IMAGE_INSTALL_append = " vim " (1)
1 Note the spaces before and after the package name. This option dumbly appends a string to an install list, so we wrap it in spaces to make sure we don’t alter the list in unexpected ways.

You can get a list of all the available packages in the layers you have configured with bitbake-layers show-recipes

Bitbake

Now you’re ready to build an image.

bitbake core-image-minimal
compiling

This step will take a while. If you used the build mirror, it might be as little as 10-15 minutes. Building everything from scratch, it will likely take a few hours.

Run the built image with QEMU

The build process creates disk images as an artefact. You can then directly run them with QEMU. (If you don’t already have it installed, install it with apt install qemu or similar.) The meta-updater layer contains a helper script to launch the images:

../meta-updater/scripts/run-qemu-ota [image name] [mac address]

Both arguments are optional; image name defaults to core-image-minimal, and if a mac address isn’t specified, a random one is generated.

Persistent storage

By default, QEMU will run your image in snapshot mode, discarding any changes you made to the disk image as soon as it exits. If you want to have a persistent VM, you need to create an overlay storage image in qcow2 format. The helper script can also manage this for you, making it easy to create an emulated fleet of devices:

../meta-updater/scripts/run-qemu-ota --overlay mydevice.cow

If the specified overlay image doesn’t yet exist, it will be created first, or launched directly if it does exist.

You should see your new device appear in OTA Connect shortly after it boots. It will generate a random name for itself during autoprovisioning; you can change the name later.

Once you’ve built your first image, try adding some new software and push your update to the OTA Connect server.