Running aktualizr with fault injection
To test the system in adverse conditions, it can be useful to make aktualizr fail in a controlled fashion.
libfiu provides a framework to do that, and aktualizr supports a number of controllable fail points.
libfiu needs to be installed on the target machine. For local compilation on a Debian-derived system, the distribution package can be used:
apt install fiu-utils libfiu-dev
Fault injection must then be enabled at CMake configure time with the
-DFAULT_INJECTION=on option (refer to general building instructions for more details).
fiu-ctrl can now be used on the newly compiled aktualizr binary to inject faults (refer to corresponding man pages).
For example, when using the fake package manager:
fiu-run -c 'enable name=fake_package_install' aktualizr -c . once
Our wrapper script in
./scripts/fiu can be used in place, with the added feature of passing strings in failinfo parameters.
Usage is as follow:
./scripts/fiu run -c 'enable name=fake_package_install,failinfo=reason' -- aktualizr -c . once
Please try to keep this list up-to-date when inserting/removing fail points.
fake_package_download: force the fake package manager download to fail
fake_package_install: force the fake package manager installation to fail
fake_install_finalization_failure: force the fake package manager installation finalization to fail
secondary_putmetadata: force virtual Secondary metadata verification to fail
secondary_putroot: force virtual Secondary Root rotation to fail
secondary_sendFirmware_xxx(xxx is a virtual Secondary ECU ID): force a virtual Secondary firmware send to fail
secondary_install_xxx(xxx is a virtual Secondary ECU ID): force a virtual Secondary installation to fail
It is encouraged to use fail points to help unit testing sad paths. Tests that require fault injection should only be run if the
FIU_ENABLE macro is defined.
The aktualizr application docker image is compiled with fault injection support. It can be used as a quick way to simulate intermittent installation failures.
First build the image:
It will tag an image
advancedtelematic/aktualizr-app:latest which includes the common aktualizr tools, as well as the fiu tools (
Then, prepare an environment for a simulated device, following the article Simulate a device without building a disk image.
aktualizr will be run through docker instead of the local system, so all commands will start with:
docker run -u $(id -u):$(id -g) -w $PWD -v $PWD:$PWD advancedtelematic/aktualizr-app
It runs the docker image inside a container with the same permissions as the local user, with a volume mounted in the current local directory.
Then, let’s launch aktualizr with
docker run --name aktualizr-fiu --rm -u $(id -u):$(id -g) -w $PWD -v $PWD:$PWD advancedtelematic/aktualizr-app fiu run -- aktualizr -c .
You can try to install a package now, which will succeed. To make all subsequent installations fail with the reason "TEST_FAILURE", use:
docker exec aktualizr-fiu fiu ctrl -c 'enable name=fake_package_install,failinfo=TEST_FAILURE' 1
To make installations succeed again:
docker exec aktualizr-fiu fiu ctrl -c 'disable name=fake_package_install' 1