Running aktualizr with fault injection
To test the system in adverse conditions, it can be useful to make aktualizr fail in a controlled fashion.
libfiu provides a framework to do that, and aktualizr supports a number of controllable fail points.
libfiu needs to be installed on the target machine. For local compilation on a Debian-derived system, the distribution package can be used:
apt install fiu-utils libfiu-dev
Fault injection must then be enabled at CMake configure time with the
-DFAULT_INJECTION=on option (refer to general building instructions for more details).
fiu-ctrl can now be used on the newly compiled aktualizr binary to inject faults (refer to corresponding man pages).
For example, when using the fake package manager:
fiu-run -c 'enable name=fake_package_install' aktualizr -c . once
Our wrapper script in
./scripts/fiu can be used in place, with the added feature of passing strings in failinfo parameters.
Usage is as follow:
./scripts/fiu run -c 'enable name=fake_package_install,failinfo=reason' -- aktualizr -c . once
Please try to keep this list up-to-date when inserting/removing fail points.
fake_package_download: force the fake package manager download to fail, optionally with a failure code supplied via
fake_package_install: force the fake package manager installation to fail, optionally with a failure code supplied via
secondary_install_xxx(xxx is a virtual secondary ecu id): force a virtual secondary installation to fail, optionally with a failure code supplied via
fake_install_finalization_failure: force the fake package manager installation finalization to fail
It is encouraged to use fail points to help unit testing sad paths. Tests that require fault injection should only be run if the
FIU_ENABLE macro is defined.
The aktualizr application docker image is compiled with fault injection support. It can be used as a quick way to simulate intermittent installation failures.
First build the image:
It will tag an image
advancedtelematic/aktualizr-app:latest which includes the common aktualizr tools, as well as the fiu tools (
Then, prepare an environment for a simulated device, following the article Simulate a device without building a disk image.
aktualizr will be run through docker instead of the local system, so all commands will start with:
docker run -u $(id -u):$(id -g) -w $PWD -v $PWD:$PWD advancedtelematic/aktualizr-app
It runs the docker image inside a container with the same permissions as the local user, with a volume mounted in the current local directory.
Then, let’s launch aktualizr with
docker run --name aktualizr-fiu --rm -u $(id -u):$(id -g) -w $PWD -v $PWD:$PWD advancedtelematic/aktualizr-app fiu run -- aktualizr -c .
You can try to install a package now, which will succeed. To make all subsequent installations fail with the reason "TEST_FAILURE", use:
docker exec aktualizr-fiu fiu ctrl -c 'enable name=fake_package_install,failinfo=TEST_FAILURE' 1
To make installations succeed again:
docker exec aktualizr-fiu fiu ctrl -c 'disable name=fake_package_install' 1