Tuesday, October 16, 2012

Unit Testing with Structural Coverage

I don't love spending time testing, so I try to make the best use of my time. I write Unit Tests when I can, and automate any other testing if possible. Sometimes I try to even make writing tests easier on myself by using some techniques I picked up from The Pragmatic Programmer: From Journeyman to Master
Side Note: If you haven't read this book, stop what you're doing, go buy it, rent it from a library, borrow it from a friend I don't care, READ IT, LEARN IT. It's one of the best books I've read about software development hands down, lots of good ideas to improve your craft.

Unit Testing

Unit Testing code is helpful for looking at code you've written in a very controlled environment, you know the inputs and outputs, and being able to duplicate your results should be pretty consistent and easy. Often times this testing puts a single function or procedure through the paces, but depending on what level you are will determine what number of units will be involved.


After writing some code a few months back, I created what I thought were some good Unit Tests for it. They all passed in the end (and before they did, we found some issues when they weren't passing) which is great. The change got closed and merged into the baseline. I thought I had made a good attempt at ensuring that the software was solid. For the data I used I agree it is solid. Then a bug was found. I hadn't tested every case (even though I thought I had). What would have been a huge help is to have simply combined Structural Coverage with it.

Structural Coverage

Structural coverage is attaching special code to test for running through each path of your code. This is helpful for simply ensuring you've tested each route. This isn't a substitute for proper testing. I'll say it again. A structurally covered program is NOT a bug free program. It's simply a program that has been proven to work as expected for a (pseudo) normal scenario on each path encountered.

Combined Testing

By combining Unit Testing and Structural Coverage, you can prove that the Unit Tests at least are checking each case once. I have since re-ran the unit tests with Structural Coverage, and guess what? I missed a few paths, this didn't explain my particular problem, however it made it easier to see other possible problems I may have in the future. I have since added some unit tests and have made sure that my Unit Testing at the minimum covers each case.No program is bug free, but all we can do is analyze and pick the Unit Tests that we as Engineers believe will cover what's necessary.

If you don't do Unit Testing where you work, I highly recommending starting to do it on your own. From the start make it as easy to setup and run as possible, this will make sure that you will use it, and that your colleagues will use it. Additionally, look for opportunities to make the unit testing automated. If your unit tests run on the system everytime a build is ran, any bugs introduced in those pieces of code can be found quickly. This keeps you from violating the DRY principle, forcing yourself to repeat yourself testing it over and over. OR trying to fix the same thing multiple times because it's not caught at the right time.

Good luck!