Every industry has their best practices and their sinful practices. The cardinal sins are the practices that many are aware of but that are just too tempting or too easy to fall into. The embedded software industry has a number of these cardinal sins but there are seven in particular that seem to have pervaded the industry throughout the decades.
Sin #1 – Not tracking metrics
Failing to track development metrics may seem like a minor sin but metrics are an integral part of embedded software engineering. Metrics provide a means for not only tracking progress and issues but also for estimation. Engineers quite often are asked, “How long will it?” or “How much will it cost?”. Questions of time and cost should be based on empirical data not on an off-the-cuff whim that is related to how optimistic the engineer is feeling at the time. Once upon a time prior to when I tracked metrics estimates were either unrealistically large or idealistically small. Without basic metric tracking determining how much flash space a microcontroller needs is extremely difficult. How is an engineer supposed to know how much RAM/ROM a typical digital input / output or UART driver takes if it isn’t tracked?
Sin #2 – Hacking instead of designing
Over the course of the last decade or so, the concept or idea of being a hacker or even a maker has become romanticized by society. Society has embraced this concept that the software engineer should be a rogue hacker who with little design or forethought can create a revolutionary and “complete” product on very short time schedule. The fact is embedded software ENGINEERING is not a hack discipline. Engineering requires forethought and design in order to be truly successful. One of the sins that are all too often committed is the mad dash code writing with no blue print, design or flow charts that through the grace of God happens to work in the most minimalistic test cases and conditions and is deemed worthy to ship because it is “functional”. Implementation and testing should follow design and architecture. A bridge isn’t built first and designed with documentation later.
Sin #3 – Starting from scratch
Digging down into the deepest level of hardware and twiddling bits to make something happen in the real world through software is fun and exciting. Embedded software engineers want to develop everything related to the software starting at the lowest levels up through application code. The problem with doing everything and starting from scratch is that it is time consuming and costly, yet it is the first instinct and often the insistence that it be done from scratch. Embedded systems have become too complex, development times too short for the average project to start with a clean slate. Leveraging vendor code, 3rd party components, open source and other standards should be leveraged to get the job done. Why spin your own RTOS when there are so many commercially available and tested alternatives?
Sin #4 – Improper tools for the job
Imagine for a moment a plumber coming to fix pipes in your home. The plumber shows up, provides an hourly rate of $100 per hour and upon agreement returns to his truck to get his tools. When the plumber returns he pulls out his sledgehammer and needle nose pliers to do the repair work. Confused and concerned you ask, “Where is your pipe wrench?” The plumber responds, “My request for a pipe wrench was denied. It’s ok, I’ll eventually get you squared away.”
Strangely enough a situation quite similar to the plumber is happening everyday all over the world when software engineers are requesting software tools to do their jobs. Basic tools such as a lint tool are being denied because they cost a few thousand dollars. A simple look at the ROI on a software tool compared to the yearly cost for the engineer should show that software tools allow things to be done faster and at a higher quality. Yet many engineers tell me they can’t get approval to purchase the most basic software tools. Interested in a professional compiler with support tools to debug faster and generate more efficient compiled code for $1000? Better luck next year, a new $15,000 scope was just purchased for the intern.
Sin #5 – Lack of continuing education
The number of transistors in a processor doubles every 2 years. The capabilities and technologies that embedded software engineers use and develop literally change at an exponential rate. Despite the rapid changes in the industry, many companies do not plan for or encourage their engineers to attend conferences or training. Part of the lack of continuing education seems to be due to design cycle timelines and pressures. There is too much to do and a company can’t spare their engineers or the cost for them to be out of pocket for a few days. The question companies should really be asking is whether they can afford to have engineers without the latest development techniques and knowledge.
Sin #6 – Working too many hours
When a deadline is looming and additional bodies can’t be thrown at it what happens? Engineers work overtime. Working too many hours is a common problem amongst embedded software engineers. Firmware runs the world and right now there just aren’t enough firmware engineers to bring to life all the devices that society demands be built. Even worse, the rate at which society wants those devices is on ever decreasing timelines. There is always a rush of needing the software done yesterday. Working too many hours though is a sure way to burn out. Rather than speeding up delivery, delivery will only be delayed. Don’t fall into the continuous over-time trap. A fresh mind can work far faster and efficiently than one that is tired and burnt out.
Sin #7 – Failing to follow best practices
MISRA-C, CERT-C and a number of other industry standards contain the knowledge and wisdom of countable embedded software engineers. Seasoned engineers who have been there, done that and learned from not only their own mistakes but also those of others developed standards such as these. Yet there are many developers who either due to time constraints, deadline pressures or other impediments ignore best practices for embedded software. I find it fascinating how these standards are often ignored even though they are so easily enforceable using software tools.
Cardinal sins pervade every industry. Best practices are usually designed to help prevent them or at least encourage proper behavior. When deadlines are looming and the pressure is on though, the temptation to fall into these seven cardinal sins is nearly irresistible. Every engineer and company has at some point fallen prey to them. The real concern is how often and what one does to get back on track.
What sins have you seen pervading the embedded software industry?