Untested code is broken code

The other day, I sat in on a demo Jesse St. Laurent was giving. The customer had a question about whether a feature worked in a specific scenario. Since it turns out that I wrote the feature, Jesse decided to defer to me. My answer was something to the effect of, “I’m 99% certain it works, but we’ll need to test it to be sure.”  That’s when Jesse told the customer a little about me, “Mike loves tests, and he believes if you haven’t proven something works with a test, then it doesn’t work.”

I graduated from college around the time when Kent Beck and Eric Gamma published a paper titled “Test Infected: Programmers Love Writing Tests”. This paper quite literally changed my life; I became ‘infected’. When the dot com bubble burst, I found myself working in a new field and was writing navigation software for helicopters. The gravity of my work was not lost on me; if I did my job wrong and someone flew a Medivac to the wrong place, then people could die.  After three years there and having returned to working for startup companies where I felt I belonged most, these lessons still stuck with me.

A decade after I first read “Test Infected”, and started working with Jim King and Mike Healey. I knew I had found like-minded, highly talented engineers who also embraced the test everything culture. Jim went so far as to update our engineering job descriptions to include, “and if you believe untested code is broken code, we want to talk to you!”

Code Coverage

At CloudTruth, we take your infrastructure seriously. That’s why we stick to high standards of unit test coverage. Our core engine sits at just under 96% covered—which means that we have a repeatable test that verifies correctness for that portion of the code. And it keeps getting better—one month ago, we were just under 95% covered.

On the above chart, green is equal to good so what does the big spot of red mean? The red area represents a small hole we have. Having this data means we are now able to assess risk and determine when to double back and fix that testing gap.

While unit tests are fantastic, we don’t stop there. Every time we make a code change, we run a battery of tests against a scaled down version of our product to ensure that not just the units but the integrated whole function in a live setting as expected. These tests take time and resources, but we can always be confident we haven’t regressed behavior.

That feature the customer asked about during the demo? While Jesse finished his demo, I ran a quick manual test to make sure it worked. On my to-do list this morning is writing an automated test to prove it. Now I know what the customer wanted to do will work and if I break it, a test will break.