Technical Quality is an Insurance Policy

| 7 Comments

Thanks to Jeff Thalhammer for this guest post. Jeff has specialized in Perl software development for over 10 years. He is the senior engineer and chief janitor at Imaginative Software Systems, a small software consultancy based in San Francisco. Jeff is also the creator of Perl-Critic, the leading static analysis tool for Perl.

Andy Lester introduced me to the concept of technical debt several years ago, and I immediately fell in love with the idea. At that time I was working in the financial industry, so the debt metaphor was useful for explaining technology issues in terms that our business stakeholders could easily understand.

I wanted to take the technical debt metaphor even further. For several years, I tried to find more analogies between debt instruments and technology. I wanted to model software development in terms of credit risk, term structure, secured versus unsecured debt, and so on. We had powerful tools for pricing financial debt so surely we could do the same for technical debt. We just needed to translate the concepts from one domain to the other. Or so I hoped.

But the more I thought about these things, the more I realized how flawed the technical debt metaphor really is. The biggest problem is repayment. Financial debt is based on the presumption of repayment. Yes, there is a risk of default and that gets priced into the interest rate. But every creditor assumes that the debtor at least intends to pay them back. With technical debt, this just isn't true. You may incur some technical debt by tightly coupling classes, or omitting test cases, or copy-and-pasting code. But if that code never needs to be changed or the project is canceled unexpectedly, then you never have to repay that debt.

And this is exactly why our software projects incur "debt" in the first place—because we are hoping that we won't have to pay it back. Sometimes we may be right, and often times we are wrong. So when I think about it that way, software development starts to look more like betting instead of borrowing. And that leads me to a new (and hopefully better) metaphor: technical insurance.

Consider car insurance. You pay a small premium now in exchange avoiding a larger payment later, if (and only if) you wreck your car. If you don't buy the insurance, then you are stuck with the full cost of repairing or replacing your vehicle, but only if you wreck.

Technical insurance works exactly the same way. For example, you can spend a little effort to write automated tests now in exchange for not crashing your production system later if a certain bug is introduced. And if you choose not to write the tests, then you lose all the revenue during the downtime, but only if that bug actually does get introduced.

Generally speaking, the price of insurance is based on the magnitude and probability of loss. If you drive an expensive car and have had several accidents, your insurance premiums will be higher than average. Likewise, if your software project is mission critical and has a history of failures, then your technical insurance will cost more. Younger, inexperienced drivers tend to have more accidents so their car insurance costs more. Similarly, a team of junior developers tend to make more mistakes and their technical insurance will cost more.

So how do we pay our technical insurance premiums? By writing tests, pair programming, doing code reviews, refactoring, or any other practice that improves code quality! Bear in mind that not all premiums have the same value—an hour spent in a code review may provide you with more insurance coverage than an hour spent writing documentation. Nor do all your premiums go toward the same policy—automated testing covers you against a different set of calamities than pair programming.

Given the premiums for your team, you then have to decide whether or not to buy a technical insurance policy. This depends on your team's level of risk aversion. If your team is willing to risk complete failure and get everyone fired, then you don't need any technical insurance. On the other hand, if the project absolutely must succeed (for some definition of success), then you need lots of technical insurance. Most teams will fall somewhere in the middle, but each will be different. You don't want too much or too little insurance—you want just enough to complement your level of risk aversion.

The technical debt metaphor still has value, however. The notion of interest—especially compounding interest—really helps people understand that putting things off can lead to bigger and bigger costs over time. A similar example in the insurance world might be medical insurance. It is a lot cheaper to buy medical insurance when you are young and healthy. But as you get older and develop "pre-existing" conditions, the cost of insurance skyrockets. The same is true for software. A new project is cheap to insure, and if it stays healthy then the premiums stay low. But if problems start to accumulate and fester, it gets more and more expensive to insure the project.

At this point, I've come to believe that software developers and managers have a lot more to learn from insurance actuaries than they do from bond traders. For me, the technical debt metaphor was inspiring, but I think that a technical insurance metaphor could be both more accurate and more useful. Smart development teams measure their unique risk exposures, know their own level of risk aversion, and carry the right types and levels of technical insurance for their specific needs.

7 Comments

Here are the slides for the talk I gave on technical debt at a YAPC years ago that I think Jeff is referring to in the first sentence.

http://petdance.com/perl/technical-debt/

Here are the slides from a talk I gave on technical debt years ago at a YAPC in Chicago. I think that talk is what Jeff is referring to in his first sentence.

http://petdance.com/perl/technical-debt/

When I explain automated tests I use the insurance metaphor so I agree with you on that.

OTOH I somehow fail to see how "the thing we used to call technical debt" or "friction" is similar to insurance. It might be similar to the "risk" we have.

So what can we do against this "risk" or "debt" or "friction".
We can write tests - mostly acceptance tests - to ensure the business gets what it wants. The internal quality of the code is irrelevant to the business if the acceptance tests all pass.

So when does internal quality matter? When the business wants to fix bugs or extend a feature. Then a higher internal quality makes the future costs lower. You are of course right that this might never happen. So there is a big "if" in there. That's what "futures" are for, aren't they? Which of course are some kind of insurance-like tools.

Excellent metaphor!

I want posters saying "If you don't buy the insurance, then you are stuck with the full cost of repairing or replacing your vehicle, but only if you wreck." and "A new project is cheap to insure, and if it stays healthy then the premiums stay low."

The debt analogy works better if you consider revolving debt like credit card debt. In this case, the lenders would make the most financial gain if the debtor never paid back the principle and there is no need to pay back the principle as long as you pay the minimum monthly fee. Using the technical debt analogy, friction would be the minimum monthly and there would be no need to ever eliminate the debt.

Here's a similar point of view that sees bad code as an unhedged call option rather than technical debt.

@szabgab: You are right: technical "insurance" and "debt" are not the same thing. However, I see them as alternative metaphors that can help us understand how the choices we make today can affect us in the future. Of course, each metaphor has it's own strengths and weaknesses.

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

affiliated with ModernPerl.net

Categories

Pages

About this Entry

This page contains a single entry by chromatic published on November 10, 2011 10:55 AM.

What the Perl 5 Compiler Modules Could Have Been was the previous entry in this blog.

Modern Perl Book Draft Period Ending Soon is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.


Sponsored by Blender Recipe Reviews and the Trendshare how to invest guide

Powered by the Perl programming language

what is programming?