4 Tactics to Unit Test RTOS Tasks

More than 50% of all embedded software projects use Real-time Operating Systems (RTOS). Unfortunately, using an RTOS can pose several problems to developers looking to use modern development techniques such as Test-Driven Development (TDD), DevOps, or automated test harnesses. For example, the first problem developers encounter when they try to write tests for their tasks is that the task function contains an infinite loop! Any test that calls the task function directly will, therefore, never be completed. This post will explore several tactics to unit test RTOS tasks which include:

  • Loop redefinition
  • Completion Signaling
  • Task Exclusion
  • Using host threading through an OSAL (Highly Recommended)

Note: For this post, we will treat Tasks and Threads synonymously. We will also use ThreadX as the example RTOS.

The Anatomy of a Task

 There are several sections in an RTOS task used to manage task behavior. First, an initialization section declares variables, instantiates objects, initializes drivers, and is responsible for casting any data passed to the task into the correct type. Next, there is an infinite loop that performs all the tasks’ behavior. For example, if we wrote a task to manage sensors, we would expect the task infinite loop to retrieve and process the sensor data periodically. Finally, a task completion section governs what happens if the task completes.

A typical task using ThreadX might look something like the following code snippet:

void Task_Sensors(ULONG ThreadInput)
{
    // SECTION 1: Initialization
    (void) ThreadInput;

    SensorData_t SensorRawData;
    SensorData_t SensorData;
    SensorData_t pSensorDataTx = &SensorData;

    Sensor_Init();

    // SECTION 2: Tasks main function / behavior / purpose
    while(true)
    {
        SensorRawData = Sensor_Sample();
        SensorData    = SensorProcess(SensorRawData);

        (void)tx_queue_send(SensorTxQ, (void *)&pSensorDataTx, TX_WAIT_FOREVER);

        tx_thread_sleep(TASK_SENSORS_PERIOD_MS);        
    }

    // SECTION 3: TasK Completion Activities
}

I have found that developers who write periodic tasks expect their tasks to run indefinitely and don’t consider what happens if the task runs to completion. Therefore, the final section is often missing in action.

Looking at this task, you can see that if a developer wants to make a call to Task_Sensors in a test harness, they are going to run into several issues. So, let’s look at the problems and follow the various tactics I often see developers attempt before reaching the most straightforward and best solution.

Loop redefinition

The first tactic that I often see engineers deploy is loop redefinition. Loop redefinition is where the infinite loop in a task is manipulated based on whether the code is production or test code. For example, the Task_Sensor code would be rewritten to something like the following:

void Task_Sensors(ULONG ThreadInput)
{
    // SECTION 1: Initialization
    (void) ThreadInput;

    SensorData_t SensorRawData;
    SensorData_t SensorData;
    SensorData_t pSensorDataTx = &SensorData;

    Sensor_Init();

    // SECTION 2: Tasks main function / behavior / purpose
    while(LOOP_STATE)
    {
        SensorRawData = Sensor_Sample();
        SensorData    = SensorProcess(SensorRawData);

        (void)tx_queue_send(SensorTxQ, (void *)&pSensorDataTx, TX_WAIT_FOREVER);

        tx_thread_sleep(TASK_SENSORS_PERIOD_MS);        
    }

    // SECTION 3: TasK Completion Activities
}

The idea is that developers can create conditionally compiled code to control whether the loop runs forever or once. The code typically looks like the following:

#ifdef PRODUCTION
    #define LOOP_STATE      true
#else
    #define LOOP_STATE      false
#endif

In addition to the compiled code, the build must also define or not define the PRODUCTION macro.

In general, this is not a great way to make an RTOS task testable for several reasons. First, the code that we are testing is changing. Our task will behave differently during testing than it does during run-time. Second, we are adding conditionally compiled code which generally is not very clean. Third, there is an opportunity for human error to occur in the definition of the macros. Finally, while loop redefinition might seem attractive for Task_Sensor, testing task interactions can become overly complicated. In fact, what happens if we need the loop to run three or four times? We now need unique definitions defined for LOOP_STATE.

Completion Signaling

Completion signaling is a modification of the loop redefinition idea to allow the task to run indefinitely until it receives a signal that the task should stop executing. In this case, the task code is modified to remove the macro definitions and instead use a loop variable as shown below:

void Task_Sensors(ULONG ThreadInput)
{
    // SECTION 1: Initialization
    (void) ThreadInput;
    bool   isRunning = true;

    SensorData_t SensorRawData;
    SensorData_t SensorData;
    SensorData_t pSensorDataTx = &SensorData;

    Sensor_Init();

    // SECTION 2: Tasks main function / behavior / purpose
    while(isRunning)
    {
        SensorRawData = Sensor_Sample();
        SensorData    = SensorProcess(SensorRawData);

        (void)tx_queue_send(SensorTxQ, (void *)&pSensorDataTx, TX_WAIT_FOREVER);

        tx_thread_sleep(TASK_SENSORS_PERIOD_MS);        

        isRunning = Task_GetDesiredRunState(TASK_SENSOR);
    }

    // SECTION 3: TasK Completion Activities
}

As you can see, at the end of the task, we check to see if the task should still be running. This solves the issue of running multiple loops and cleanses the code by removing the macros. However, we have now introduced complexity into how a task runs and opens the opportunity for memory corruption or a single event upset (SEU) to change the state of isRunning and complete our task.

A task running to completion in production might not seem like a big deal, but not all RTOSes handle this gracefully. For example, if you allow a FreeRTOS task to run to completion, the kernel will choke and stop executing all code!

Task Exclusion

Another interesting method to test an RTOS task is to use task exclusion. Task exclusion occurs when we don’t test our tasks! Instead of trying to call the task from a test harness, we create tests that run the code that would be in the task itself.  Task exclusion requires us to rewrite our functions to look like the following:

void Task_Sensors(ULONG ThreadInput)
{
    // SECTION 1: Initialization
    (void) ThreadInput;

    Task_SensorInit();

    // SECTION 2: Tasks main function / behavior / purpose
    while(true)
    {
        Task_SensorRun();

        tx_thread_sleep(TASK_SENSORS_PERIOD_MS);        
    }

    // SECTION 3: TasK Completion Activities
}


/**********************************
 * Placed in a different module
 **********************************/

void Task_SensorInit(void)
{
    SensorData_t SensorRawData;
    SensorData_t SensorData;
    SensorData_t pSensorDataTx = &SensorData;

    Sensor_Init();
}

void Task_SensorRun(void)
{
    SensorRawData = Sensor_Sample();
    SensorData    = SensorProcess(SensorRawData);

    (void)tx_queue_send(SensorTxQ, (void *)&pSensorDataTx, TX_WAIT_FOREVER);
}

Honestly, the above code is how the task should look. The code is very clean and easy to follow. The problem, though, is that we got here by trying to use a less-than-optimal technique.

The task code we now see in Task_Sensors is so simple that task exclusion would say we don’t need to test it. So instead, we would test the Task_SensorInit and Task_SensorRun functions within our test harness. After all, these functions are saved in a separate module, so we can exclude task code from the test harness and achieve our desired 100% code coverage, right?

The problem with task exclusion is that we never test the task code until we run the application on the target. Unfortunately, we also trick ourselves into believing that our tests cover all our code.

The advantage is that we can call the functions within the tasks as needed from the test harness. We have achieved this and avoided needing to deal with the infinite while loop. The code is cleaner, and we have not created a bunch of conditional compilation statements.

While this technique can make it easier to test task code, it technically isn’t testing the task code. Instead, it’s a workaround. To test your task code, you should try to create a thread or task on your host machine and run the code there.

Using host threading through an OSAL (Highly Recommended)

The real problem with testing RTOS tasks has nothing to do with the fact that most tasks have a while statement. Instead, the issue stems from how the developer thinks about and approaches testing. All the tactics that we have examined up to this point assume that we want to call Task_Sensors directly from our test harness. That is where the problem lies. In our test harness, we want to create a Task_Sensors task or thread that runs, not make a function call!

The root cause for our testing woes is that developers don’t leverage operating system abstraction layers (OSAL). Instead, they go directly to the RTOS APIs and use those. While calling the RTOS APIs is quick and seems like a good approach, it is tightly coupling the RTOS to the application code. That coupling makes it very difficult to test application code that uses a task or thread properly.

The optimal approach is to hide the RTOS details behind an Operating System Abstraction Layer (OSAL). For example, if you examine the various versions of Task_Sensors we’ve looked at so far, you’ll find that we are using the ThreadX tx_queue_send API to send a message to a queue. Therefore, we should have placed those details behind an OSAL so that our task looks like the following:

void Task_Sensors(ULONG ThreadInput)
{
    // SECTION 1: Initialization
    (void) ThreadInput;
    bool   isRunning = true;

    SensorData_t SensorRawData;
    SensorData_t SensorData;
    SensorData_t pSensorDataTx = &SensorData;

    Sensor_Init();

    // SECTION 2: Tasks main function / behavior / purpose
    while(true)
    {
        SensorRawData = Sensor_Sample();
        SensorData    = SensorProcess(SensorRawData);

        (void)OSAL_Q_Send(SensorTxQ, (void *)&pSensorDataTx, OS_WAIT_FOREVER);

        tx_thread_sleep(TASK_SENSORS_PERIOD_MS);        
    }

    // SECTION 3: TasK Completion Activities
}

OSAL_Q_Send is an abstraction our application code uses to send data in a queue. The application should not care whether we are using ThreadX, FreeRTOS, pthread, or any other RTOS or task scheduler. An implementation file is provided based on which system we want to compile the code for. There are a lot of benefits to this, such as:

  • The application is not coupled to the RTOS
  • The test harness can use the hosts threading framework
  • The application is portable and reusable
  • We avoid ad-hoc and hacky testing methodologies for testing.

For many developers interested in using DevOps and automated test harnesses, you’ll likely at least have an implementation for your chosen RTOS and pthread, the POSIX thread library for Linux. Unfortunately, how we use pthread and design and build an OSAL are beyond today’s scope, but we will explore those topics in the future.

For now, if you are interested in looking at some OSAL examples, I recommend taking a look at CMSIS-RTOS-V2 and NASA’s Aeronautics GSC-18730-1. Chances are, these are way overkill for what you need, but they are good examples of fully implemented OSALs. I’d recommend exploring them and slowly designing and building your OSAL that you can use across all your applications.

Unit Test Tactics Conclusions

There are several tactics that developers can use to unit test a task or thread. As we have seen in today’s post, most tactics developers can deploy are workarounds for failing to architect their tasks to use an OSAL. Once an OSAL is in place, task code can be tested using any RTOS or local thread library by providing the necessary functions behind the abstraction layer. The OSAL layer helps:

  • Simplify the test tactics
  • Keeps the code clean
  • Improves flexibility, portability, and reuse

If you want to test all your code, then leveraging pthread through an OSAL is the best approach.

Additional Resources:

References:

Share >

12 thoughts on “4 Tactics to Unit Test RTOS Tasks

  1. A useful article, thanks! I think the conditional compilation example should be a do-while loop instead, if you want it to run once, as opposed to never. I find wrapping in an OSAL has other benefits: I’ve added consistent error handling (as previously, some direct calls to the OS ignored or handled errors in different ways).

  2. OSAL as an “Abstraction Layer” may be a costly exercise amounting to designing another OS API. This may be reasonable and beneficial if the application is using limited portion of the RTOS API. However I would advise to call it OS Separation Layer and structure accordingly, the goal here is not to “abstract” the OS, the goal is to provide a layer which simplifies writing tests by centralizing all the stuff external to the application code which would need to be stubbed for testing.

    I attempted once to write (limited!) OSAL covering ThreadX, FreRTOS and uC/OS-II (Micrium). The scope was limited but it attempted to cover queues, timers and message buffers. It was not pretty: on the 1st glance all these three OSes look quite similar however when I got to actual implementation I found many subtle differences mostly conspiring to complicated memory allocation. The main goal of that project was performance comparison. The main benefit was realization that each of them will do, but for maximum performance certain architectural decision SPECIFIC for each OS would have to be made …

    • Thanks for the comment.

      It can be not easy to get something that covers multiple OS’s, but it is doable. If something is needed for multiple systems, it may be worthwhile to adopt POSIX instead.

  3. Hello Jacob,
    why you don’t mention mocks/stubs ?
    From my perspective even with an OSAL it will give me the freedom to execute all lines in the task ?

    Best regards
    Alex M.

    • Thanks for the question.

      The short answer is that if I covered everything in this one post, the blog would just be too long.

      I plan to do another post in the near future something like “4 More Tactical Techniques to Test RTOS Tasks” that would cover test doubles, fakes, mocks, and stubs. Granted, even then, there might be enough material there that it would need to be split up across several blogs.

  4. A useful article, thanks! I think the conditional compilation example should be a do-while loop instead, if you want it to run once, as opposed to never. I find wrapping in an OSAL has other benefits: I’ve added consistent error handling (as previously, some direct calls to the OS ignored or handled errors in different ways).

  5. Doesn’t seem like the problem of the while loop has been solved or have I missed something? Another way apart from the OSAL, which is a good idea is to add a message queue (which most tasks use anyway) and send a message to end the task.

    • I might have been subtle in the solution.

      The abstraction layer allows a developer to create test cases where they actually create threads and run the task on their system. The test harness can monitor what is going on and then kill the threads when the test has completed.

      Thanks for the question! (And yes, a message queue could also be used!)

Leave a Reply to Jacob Beningo Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.