During development, a very common activity is to create new C modules. Creating a new C module often comes with a bunch of repetitive work. For example, if you are creating a Gpio module, you likely have header and source comment text to copy and paste into the new files to look consistent. Undoubtedly, you must go through those files and put who the developer is, when you created the module, and so forth. Some of this can be automated using templates within IDEs like Ecplise, but even then, customization is required.
This post will show you a simple script to automate C module creation using Python. The script is not the one I use on a day-to-day basis, but it demonstrates some cool features of Python and how you can use it to automate common activities. I’ve also posted the code to my public GitHub account here.
Why Automate?
In software development, we often watch for code that wants to duplicate itself. When we see code duplication occur, we deem that code to be unclean and refactor it. When we refactor the code, the code is cleaner, and we have created functionality that can be reused to keep it clean. Unfortunately, I’ve encountered a lot of developers who don’t refactor their development processes! They manually repeat the same mechanical process over and over.
Automation isn’t the be-all solution, but it does have many benefits. For example, automating common code activities can:
- Decrease development time
- Minimize human error (bugs and defects)
- Remove repetitive activities
- Open up new avenues for innovation
- Keep the code clean
For the longest time, I used templates in my development efforts to help me minimize rework and get more done. Unfortunately, it took me a little while to realize that my templates, with a little bit of scripting, could become even more powerful and save tons of time. Today, I have a library of scripts that automate many of my development processes, which helps me save a ton of time and allows me to focus on high-value activities.
Leveraging Python for Embedded Software Automation
Python is a great programming language. Python is easy to learn and has a ton of libraries that can be used to simplify nearly any programming activity. It’s so easy to use that I’ve encountered elementary students who can program with it. There are also tons of free resources available on the internet to help developers quickly accomplish their needs.
The Python string template class allows a developer to create a string template text file with data strings that are ready for string substitution. For example, I might have an author of a module that will change based on the coder’s identity. In a template file, I can simply write something like:
Author: ${author}
to create a substitution tag. The ${author} tag is a placeholder for string text that will be replaced using the string libraries substitute method.
To avoid copying and pasting a header and source module template text into every module we want to create, we can modify the template to use tags and then script the substitutions we want. Ultimately, when I create a new module, instead of copying, pasting, and editing, I should be able to open a terminal and type:
python generate.py Gpio OutputFolder
The Python script, generate.py, is told that we want to create a module named Gpio and that the module will be stored in the OutputFolder. OutputFolder, in this case, is an optional argument that, if not provided, generates the files in the directory from which the script was executed.
With just these few details, we can create a quick and dirty Python script that reads in a template file and use string substitution to create our module. Keep in mind that I’m going to show you the minimum required. You can easily modify and build out these scripts to generate quite a bit of code.
An Example Template Header File
My personal preference is to document my modules using Doxygen. Over the years, I have developed several Doxygen templates for header and source modules. Some might argue that using Doxygen and commenting code modules violates clean coding practices. As a consultant, I nearly always provide API documentation with any code I write and have found it useful when it’s also provided to me. So while it may be considered “code clutter” in some coding circles, I’ve generally found it useful. Why does this matter? It matters because the template header file I am about to show you is documented in this manner.
First, let’s take a look at the header template. You can find the downloadable copy here.
/****************************************************************************
* Title : ${module}
* Author : ${author}
* Origin Date : ${date}
* Notes : None
*
* THIS SOFTWARE IS PROVIDED BY ${company} "AS IS" AND ANY EXPRESSED
* OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL ${company} OR ITS CONTRIBUTORS BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
* IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGE.
*
*****************************************************************************/
/****************************************************************************
* Doxygen C Template
* Copyright (c) 2013 - Present -- Jacob Beningo - All Rights Reserved
*
* Feel free to use this Doxygen Code Template at your own risk for your own
* purposes. The latest license and updates for this Doxygen C template can be
* found at www.beningo.com or by contacting Jacob at [email protected].
*
* For updates, free software, training and to stay up to date on the latest
* embedded software techniques sign-up for Jacobs newsletter at
* https://beningo.com/814-2/
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Template.
*
*****************************************************************************/
/** @file ${module}
* @brief This module TODO: WHAT DO I DO?
*
* This is the header file for the definition TODO: MORE ABOUT ME!
*/
#ifndef ${fileMacro}_H_
#define ${fileMacro}_H_
/******************************************************************************
* Includes
*******************************************************************************/
#include <stdint.h>
#include <stdbool.h>
/******************************************************************************
* Module Preprocessor Constants
*******************************************************************************/
/**
* Doxygen tag for documenting variables and constants
*/
#define CONSTANT 5
/******************************************************************************
* Module Preprocessor Macros
*******************************************************************************/
/******************************************************************************
* Module Typedefs
*******************************************************************************/
/**
* This enumeration is a list of test types
*/
typedef enum
{
TEST_TEST1, /**< Test Type 1 */
TEST_TEST2, /**< Test Type 2 */
}Test_t;
/******************************************************************************
* Module Variable Definitions
*******************************************************************************/
/******************************************************************************
* Function Prototypes
*******************************************************************************/
#ifdef __cplusplus
extern "C"{
#endif
void ${module}_Init(void);
#ifdef __cplusplus
} // extern "C"
#endif
#endif /*${fileMacro}*/
/*** End of File **************************************************************/
If you took a moment to scan the above code, you’ll notice a few string tags in the header. These tags include:
- ${module}
- ${author}
- ${date}
- ${company}
- ${fileMacro} – all upper case module name
We will provide what these tags should replace when discussing the Python script.
One additional item to note is that this header has only a single function prototype. There is only one function prototype to keep the example simple. You can add additional prototypes like:
void ${module}_Create(void);
void ${module}_Destroy(void);
uint32_t ${module}_Read(void);
uint32_t ${module}_Write(void);
Your homework is to create your own template and fill out your default module interfaces.
An Example Template Source File
The source template that we will use is also designed for Doxygen. It’s a little bit simpler than the header. You can download the source template here. You’ll notice with these templates that they have weird extensions. I defined these extensions as:
.htf – header template file
.ctf – source (c) template file
My sample source template file can be seen below:
/*******************************************************************************
* Title : ${module}
* Author : ${author}
* Origin Date : ${date}
* Notes : None
*
* THIS SOFTWARE IS PROVIDED BY ${company} "AS IS" AND ANY EXPRESSED
* OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL ${company} OR ITS CONTRIBUTORS BE LIABLE FOR ANY
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
* IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGE.
*
*******************************************************************************/
/****************************************************************************
* Doxygen C Template
* Copyright (c) 2013 - Jacob Beningo - All Rights Reserved
*
* Feel free to use this Doxygen Code Template at your own risk for your own
* purposes. The latest license and updates for this Doxygen C template can be
* found at www.beningo.com or by contacting Jacob at [email protected].
*
* For updates, free software, training and to stay up to date on the latest
* embedded software techniques sign-up for Jacobs newsletter at
* https://beningo.com/814-2/
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Template.
*
*****************************************************************************/
/** @file ${module}
* @brief This is the source file for TODO: WHAT DO I DO?
*/
/******************************************************************************
* Includes
*******************************************************************************/
#include <stdint.h> /* For portable types */
#include "${moduleInclude}.h" /* For this modules declarations */
/******************************************************************************
* Module Preprocessor Constants
*******************************************************************************/
/**
* Doxygen tag for documenting variables and constants
*/
#define CONSTANT 5
/******************************************************************************
* Module Preprocessor Macros
*******************************************************************************/
/******************************************************************************
* Module Typedefs
*******************************************************************************/
/******************************************************************************
* Module Variable Definitions
*******************************************************************************/
/******************************************************************************
* Function Prototypes
*******************************************************************************/
/******************************************************************************
* Function Definitions
*******************************************************************************/
// TODO: UPDATE AND COPY THESE FOR EACH NON_TRIVIAL FUNCTION
/******************************************************************************
* Function : ${module}_Init()
*//**
* b Description:
*
* This function is used to initialize the ${module} based on the configuration
* table defined in dio_cfg module.
*
* PRE-CONDITION: Configuration table needs to populated (sizeof > 0)
*
* POST-CONDITION: A constant pointer to the first member of the configuration
* table will be returned.
*
* @return A pointer to the configuration table.
*
* b Example Example:
* @code
* const Dio_ConfigType *DioConfig = Dio_GetConfig();
*
* Dio_Init(DioConfig);
* @endcode
*
* @see ${module}_Init
*
*******************************************************************************/
void ${module}( void )
{
}
/*************** END OF FUNCTIONS ***************************************************************************/
If you took a moment to scan the above code, you’ll notice a few string tags in the source. These tags include:
- ${module}
- ${author}
- ${date}
- ${company}
These templates don’t completely remove the need for minor editing, but they get far closer than one would otherwise.
The Module Generator Class
At the end of the day, a developer should want to instantiate an object named module and then call the make method. Magic happens, and we end up with the desired header and source modules ready for us to add our code. For example, optimally, my generate.py script would have lines like:
Module = ModuleGenerator(author, getDate(), args.Name, company, outputPath) Module.Make()
- WriteHeader
- WriteSource
Both methods follow a similar structure:
- Read the header/source template into memory
- Substitute the tags with the desired string data
- Write the file
As an example, let’s look at WriteHeader, which can be seen below:
def WriteHeader(self):
# Read in the .h template file
with open("templates/header.htf") as inputFile:
headerTemplate = string.Template(inputFile.read())
inputFile.close()
# Substitute the new strings with the template tags
headerFile = headerTemplate.substitute( module=self.moduleName,
author=self.author,
date=self.date,
company=self.company,
fileMacro=self.moduleName.upper())
# Write out to header file
self.mkdir_p(self.outputPath)
with open(self.outputPath + self.moduleName.lower() + ".h", "w") as outputFile:
outputFile.write(headerFile)
outputFile.close()
The most interesting piece of code here is the string substitution. You can see that the string object headerTemplate has a substitute method. We pass the substitute method, the tag name, and the replacement string we want to use. It’s as simple as that!
Running the Script
With the ModuleGenerator class written, the main generate script just needs to read in a few arguments, create the class, and then invoke it. The code to do this in Python can be seen below:
import yaml
import string
import argparse
from ModuleGenerator import ModuleGenerator
from datetime import date
def getDate():
today = date.today()
return today.strftime("%m/%d/%Y")
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Create a new C module from templates')
# Required positional argument
parser.add_argument('Name', type=str,
help='Module name to create')
# Optional positional argument
parser.add_argument('opt_outputPath', type=str, nargs='?',
help='An optional output path argument')
args = parser.parse_args()
if args.opt_outputPath is None:
outputPath = ""
else:
outputPath = args.opt_outputPath + "/"
# Get the configuration for the author and company information
stream = open("config/config.yaml", 'r')
config = yaml.safe_load(stream)
author = config.get("author")
company = config.get("company")
# Generate the output modules
Module = ModuleGenerator(author, getDate(), args.Name, company, outputPath)
Module.Make()
In addition to reading the arguments, you can see that I’m using a YAML file in a configuration folder that contains the author and company information. Date information is generated from the system time.
At this point, I can run this script to create a Gpio module using:
python generate.py Gpio
Running the script produces the following, which you can see now includes gpio.h and gpio.c:
If we look at just the header information in gpio.h, we can see that our string replacement worked!
Conclusions
Automation can help developers with simple and mundane tasks like creating a new header/source module. Creating a module is simple, yet, developers often spend a lot of time doing so. Using simple Python scripts that can grow in sophistication and capabilities over time can help improve software development dramatically. We’ve seen in this post how to use Python for string substitution. A simple yet, powerful example. Yes, my example scripts aren’t polished, but I hope they give you ideas on how you can improve your own development processes and automate activities you do repeatedly day in and day out.
