In the ever-evolving world of embedded systems, there’s a pressing need to adopt best practices from the software domain to ensure timely, consistent, and quality releases. Embedded CI/CD is one of those practices that is helping to revolutionize how embedded software is developed.

Continuous Integration (CI) and Continuous Deployment/Delivery (CD) form the bedrock of modern software development practices. If you’re an embedded developer, you’ve probably heard of these practices but haven’t had the time to adopt them and update your workflows.

Today is the first post in a series about streamlining your development using CI/CD with VS Code and the NXP SDK. You’ll learn why CI/CD is essential and how to set up Github Actions with NXPs MCUXpresso for VS Code extension. Before we get into the fun technical details, we will look at fundamental CI/CD concepts and understand their value to embedded software developers.

The Value of CI/CD in the Embedded Product Development Cycle

In traditional software development, developers and teams work in isolation for extended periods, occasionally merging changes. The result was a “merge hell,” where reconciling different pieces of code could take significant amounts of time with the potential of introducing errors into the code. Isolated development wasn’t working, which is where CI/CD took the stage.

Continuous Integration (CI) emphasizes integrating code from various team members frequently (often daily). Each integration is automatically built, analyzed, and tested to ensure early detection of integration bugs.

Continuous Deployment/Delivery (CD) is a logical extension of CI. Once your code is integrated and tested, it’s automatically deployed to production (Continuous Deployment) or made ready for deployment (Continuous Delivery), reducing manual intervention, and leading to faster releases.

CI/CD has the potential to improve embedded software development practices dramatically. For embedded developers, you’ll find benefits such as:

• Faster Time-to-Market with regularly tested and ready-to-deploy code that reduces the overall development time.
• Improved Quality through automatic testing results in fewer bugs and a higher-quality product.
• Enhanced Collaboration due to teams being better aligned as they work towards a common, integrated codebase.

You might think these benefits sound great, but you’re writing embedded software with hardware dependencies! Embedded software differs and doesn’t lend itself well to CI/CD! Well, that isn’t the case. You can just as easily apply CI/CD to embedded software development using a CI/CD pipeline. Let’s explore what a typical embedded CI/CD pipeline might look like.

Embedded CI/CD Pipeline Overview

A CI/CD pipeline is the process that moves a piece of software from development into production. A typical CI/CD pipeline includes automating building, testing, analyzing, and deploying software to the end users. It’s the tool that developers, QA, and operations use to improve collaboration between various teams and gain benefits such as:

• Improving software quality
• Decreasing the time spent debugging
• Decreasing project costs
• Enhancing the ability to meet deadlines
• Simplifying the software deployment process

CI/CD should not be viewed as the only necessary component in a DevOps process. It’s simply a tool. For example, you might find an overall DevOps process like Figure 1 below. As you can see, the process maps out the interactions for developing new software features, testing them, and deploying them into the field.

Figure 1: An example Embedded DevOps delivery pipeline process (Source: Embedded Software Design; page 184).

Hidden within Figure 1 is the CI/CD pipeline. A typical CI/CD pipeline comprises three simple activities:

• Building
• Testing
• Deploying

Conceptually, pipelines are straightforward, but the details often get developers hung up. For example, if I were to list out the various components that make up a CI/CD pipeline, my short list would include the following:

• Repositories – Where the source code is pulled from for a build. (Maybe one or more).
• Automated Build System – Often a containerized system that can automatically build the software.
• Automated Testing – Includes unit, integration, system, and regression testing.
• Static Code Analysis – Analysis of the software that includes checks against coding standards, metrics analysis, language correctness, etc
• Hardware Emulation / Simulation – Use virtual hardware to test and verify the software’s correctness.
• Deployment Mechanism – A method for delivering new firmware to the product once it has successfully passed all automated tests and manual gates.
• Monitoring and Reporting – A mechanism for monitoring progress and reporting software status and any issues that may need to be investigated.
• Security Analysis – Analysis to identify security vulnerabilities and ensure the system is protected. (Security could go under static analysis, but I thought it was worth listing separately since it can be easily overlooked).

As you can see, thinking through each of these areas, identifying the implementation you want to use, and configuring it will be a lot of work. It will require forethought and time investment to get your own CI/CD solution implemented, but what should that solution look like? Every team will have its own needs, but there are some standard features that all CI/CD pipelines will have. Let’s look at a few areas and define a generic CI/CD pipeline for embedded developers. We’ll keep it simple so you can build on it in the future.

Defining a typical Embedded Pipeline

A typical embedded CI/CD pipeline will vary based on your teams’ needs. While there is a lot of customization that you can do, a first-pass CI/CD pipeline should at least contain the following:

• Build – The build job will take your firmware and generate release binaries.
• Analysis – The analysis build job will statically analyze your code. The typical analysis would include cyclomatic complexity, code metrics, and coding standard adherence such as style guide, MISRA, and/or CERT.
• Testing – The testing job will perform all the tests required to ship the product. You might include unit, functional, integration, system, and performance tests.
• Reporting – The reporting job will pull together the results from the previous jobs to provide information about build success, analysis results, test coverage and results, and more.
• Merge – The merge job will merge the new feature into the deployment branch when all the jobs are successful. There is often a human element here, but this can also be automated.
• Deployment – When the merge job completes successfully, the deployment job will run and start the field deployment process. Deployment usually interacts with fleet deployment software that can push firmware to devices in the field.

If you were to visualize the pipeline, you’d find it looks like Figure 2. Developers working with this typical process would commit code to their version control system (VCS) and generate a pull request. An automated build system, a CI server, would automatically fetch the latest code and build the software. In addition, and depending on how the pipeline is set up, automated tests and analysis would be done after the build process is completed successfully. If the tests pass successfully, the code is deployed to production or a staging environment for further review. 

Figure 2 – An ideal CI/CD pipeline that combines the essential jobs required to build and deploy modern embedded software. (Source: Embedded Software Design; page 25). 

Tools of the Trade: CI/CD Toolset

When you explore the field for tools, you’ll discover many tools available to help you build out your embedded CI/CD pipeline. These tools range from tools that cover a small portion of the CI/CD pipeline, to complete integrated solutions. The most popular tools include:

• Version Control Systems (VCS) like Git
• Build Tools such as Make and CMake
• CI Servers like Jenkins, Travis CI, and CircleCI
• Containerization tools like Docker and Kubernetes help replicate environments consistently and enable pipeline job execution.

Beyond these tools, integrated platforms like GitHub and GitLab include all these features wrapped into a single solution. The exact platform or combination of tools that you select will depend on you and your team’s needs. For this series, we will use GitHub Actions with CMake to build a simple embedded CI/CD pipeline example. I’ll be using MCUXpresso for VS Code with the NXP LPC55S69-EVK.

Conclusions

CI/CD, though often associated with web and software applications, offers undeniable advantages to the realm of embedded systems. Using platforms like NXP’s SDK and versatile IDEs like VS Code can drastically change how embedded developers approach product development. In the rest of this series, we will see how to set up GitHub Actions for a blinky application on the NXP LPC55S69-EVK. You can directly apply these skills to start your own embedded CI/CD pipeline.