3 Tips for Unit Testing Embedded Software

How do you know that your software is working the way that it is supposed to? You test it! The real question, though, is “How do you test it?”. In the old days, developers used to just manually test their software. Unfortunately, manual testing is not a great way to test software. The sheer number of test cases, the time required to do the testing, and labor intensity almost guarantee that the software will not be adequately tested.

The solution to improving embedded software testing is to use automated testing. Automated testing can come in many forms, but for now, we will focus on unit testing. Unit testing is “a software testing method by which individual units of source code—sets of one or more program modules together with associated control data, usage procedures, and operating procedures—are tested to determine whether they are fit for use”.[1] Today’s post will explore three tips for developing and running unit tests for embedded software.

Tip #1 – Organize your software into components for testing

Typically embedded software programs will be composed of several dozen modules thrown into a single program folder. Today, it’s a little more common to see some folder organization where modules are organized by application, middleware, and drivers. Program organizational structures like this are okay, but it can be much easier to manage the program by components when considering unit tests.

A component is a module that encapsulates a set of related functions[2], data, and test cases. For example, a developer that is writing an application component for an FIR filter might organize it as follows:

FIR Filter

– include

— fir.h

– source

— fir.c

– tests

— fir_test.c

Building out a folder structure like this might at first seem a bit painful. However, it keeps all the software modules required by the component to perform its purpose together and the test cases! Furthermore, organizing a component like this makes the component easy to port, or perhaps more importantly, easier to reuse in other software projects.

Tip #2 – Develop Software using Test Driven Development (TDD)

The Agile movement has provided software developers with many processes and tools designed to help them develop quality software faster. One methodology that Agile has produced is Test Driven Development, which is often referred to as TDD. TDD “is a software development process relying on software requirements being converted to test cases before the software is fully developed and tracking all software development by repeatedly testing the software against all test cases[3]“.

I was first introduced to TDD around 2014 when I attended a lecture on the topic by James Grenning at the Embedded Systems Conference. My first impression was that TDD seemed to offer a lot of potential but seemed like it was much more work, and I was not convinced it could deliver on its promises. Unfortunately, it took me several years before I could seriously dig in and integrate it into my development processes. However, once I did, I started to drink the Kool-Aid and see the technique’s value.

The full details are beyond the scope of this post, but my favorite references for TDD are Kent Beck’s book Test-Driven Development and James Grennings Test-Driven Development for Embedded C. In general, TDD changes how developers write their software by focusing on test cases. Developers create a test case, make it fail, and write the code required to pass the test case. By doing this, they are building test cases that they know will catch problems if a bug is introduced into the software.

Tip #3 – Leverage Docker and a unit test harness

The tools available for embedded developers to develop unit tests have evolved considerably over the last few years. When I first started to play with automated testing, I found that getting the tools set up was a huge challenge. This is no longer the case today.

There are several ways that teams can set up their unit testing. First, they can set up their testing as part of a continuous integration and continuous deployment (CI/CD) system. CI/CD allows teams to run their test cases automatically as part of their build and deployment processes. Next, developers can just pick a test harness and install it on their system. The test harness, in this case, is running in a stand-alone environment. Finally, developers can build out their test harness and development processes and set them up within a Docker environment. Docker allows developers to run their development environment in a portable image that minimizes setup time and improves consistency between developers.

A test harness can be set up within Docker and then easily deployed to multiple developers so that they can get their environment set up using just a few commands. It’s a compelling process that we will explore in several upcoming blogs.

Conclusions

Creating and using automated testing for embedded software seems daunting at the beginning. However, given how complex today’s systems are becoming, it is nearly impossible to perform testing by hand. The only real solution is to develop automated tests that can be used to execute all the system’s features. Unit tests are the most common tool available to developers and can dramatically improve system quality while decreasing the overall time spent developing software. In this post, we explored several tips for unit testing embedded software. In future posts, we will explore how to set up and write our test cases.

[1] https://en.wikipedia.org/wiki/Unit_testing

[2] https://en.wikipedia.org/wiki/Component-based_software_engineering#Software_component

[3] https://en.wikipedia.org/wiki/Test-driven_development

One thought on “3 Tips for Unit Testing Embedded Software”

  1. I think this misses one of the biggest tips: Write portable, well-abstracted code! By correct use of hardware and OS abstraction layers as well as using a coding standard which emphasizes portability we can ensure the majority of the application logic (and even much of the middleware) is testable on a host (or CI) machine.

    There are numerous other great things that come with it, too, including simulation (SIL) and log replay, better tool support (e.g. static analyzers and sanitizers), and fault injection capabilities via mocked drivers and OS calls.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.