For the umpteenth time in my career, I am debugging a prototype board that has exactly zero test points. I didn’t design this board, but have inherited it from a frantic colleague who is behind schedule and is having trouble communicating with an onboard SPI device. The first step in this troubleshooting endeavor is to simply take an oscilloscope or logic analyzer and check the signal lines to see if they are wiggling at all. Of course, the microcontroller pins are soldered under the chip, with no accessible vias or test points. The troublesome device offers no help at all with its pins also neatly tucked below. Where oh where are my test points!
When I talk with clients and colleagues about how much time they spend debugging their systems, I’m often given numbers that range from 20 – 50% with the occasional tongue in cheek response of 80%. As embedded system developers, we spend a lot of time trying to get our systems to work the way they are supposed to. So why are we so stubborn (or arrogant?) to not follow simple best practices that can save us time, headaches and help us deliver on-time?
When it comes to test points, I’ve generally heard two arguments for not putting them on a board. First, the security experts recommend making their lives difficult by minimizing test points, access to JTAG, etc. The harder it is to get access to the MCU and the on-board software the better. This is a good recommendation, for production hardware, but every security expert I’ve talked with has always confided that no matter what someone does, they find their way around the board to get access to what they need anyways. Best case, the would-be hacker just gets annoyed because they have extra work to do. So, in the end, this recommendation is just a short-term stop gap that is just making the development teams life a living hell.
Second, putting test points or extra vias on the board costs extra money and every good layout engineer knows that they should be minimizing costs. The problem with this argument is that it’s from the last century. Board costs are no longer dependent on how many holes have been drilled in them. The industry has become so commoditized that you can drill vias all over the place and the end cost increase is a few pennies if there is a cost increase at all. (Board costs are now more dependent upon physical size and copper weighting).
On the very first board that I ever designed, back in probably 2002, the first painful lesson that I learned was that in order to debug the hardware and software efficiently, every signal, yes, EVERY signal, needs to be easily accessible. Whether it’s through clamping onto a pin, having an exposed pad to solder a wire on, or my personal favorite, a via that I can solder a header onto. If a developer doesn’t have access to the signals and runs into problems, their system debugging turns into a time-consuming guessing game that is stressful, costs the company money, and personally is not fun at all. Developers that have test points on the other hand can measure, analyze and follow systematic methods based on observation to quickly resolve any issues and move on to the next.
Test points provide developers with insights into the signals and even the software and without them, debugging the system undoubtedly takes at least twice as long if not longer. After I learned the test point lesson from my first board, I quickly became a two-spin wonder where nearly every board I designed was ready for production testing on the second spin. Test points can always be removed for production boards after the hardware and software is working as expected if so desired. Even when they are removed, keep an exact replica of the production board with test points for in-field troubleshooting and debugging. It’s nearly guaranteed that they will be needed.
Well I have a lot of soldermask to remove (to make my own test points) and very little time so for the umpteenth time, please put test points on the board! Your fellow engineers, colleagues, consultants and software wizards will love you for it.
I just had an occaision that validated my choice to leave all my testpoints in during production. A recent design has a graphics chip which is in a QFN package, no problems for the first few hundred pieces, then we had a rash of systems with no graphic output and even sometimes no Vcc power at all. A quick resistance check on my testpoints revealed that we were getting shorts underneath the QFN package, both pin to pin and pin to the central square ground pad. An adjustment to the pastemask and our yield is now almost 100% 🙂