Tests are Code Too!


Of all of the frustrating debates in software development, bickering over the value of automated testing confuses me the most. I can understand why people who've never tried pair programming seriously have doubts about it. I can even understand why people who know nothing about programming or construction think that designing a building is like designing software.

I don't understand programmers who think automated testing isn't all that useful because tests can be difficult to maintain.

Perl's testing culture had its renaissance when several of us realized that people saw testing as confusing black magic. (No joke; the "copy and paste this header into all of your test files!" generated by h2xs in test.pl used to say "black magic here" before four lines of gobbledegook.)

The nice thing about a renaissance is that you can clear away confusing, magical thinking and replace it with bright, understandable foundations for grander, more elegant, more powerful, and futureful thought.

This occurred on two levels. First, the parallel development of Test::Simple and Test::More made two libraries that were almost trivially easy to understand. Writing good tests requires skill and experience, but writing tests at all requires two minutes of reading and experimenting. The path to mastery begins not with copying and pasting black magic gobbledegook that explicitly warns you off of understanding it but by saying what you mean to say.

Another part of that was good initial design of what would become TAP way back in 1987, as well as other parts of the toolchain, but removing the initial barrier to starting was a huge improvement.

The other foot of the renaissance was Test::Builder. You can start with Test::More and use it productively... but if you're not careful, you'll end up with long, procedural test files with a lot of duplication.

Test::Builder exists solely to remove duplication. It's a foundational library for the abstraction of tests and test structures. You can see this in Testing with Test::Class. You don't necessarily have to write your own testing modules to take advantage of this possibility; the CPAN has hundreds of Test::* modules for your own use.

That's what modern Perl programmers do. We solve problems, then we make the solutions available. We'd never have achieved this for our tests if we hadn't realized the simple, fundamental fact of automated tests: they're just code. All of the good habits you have for managing code apply to managing tests.

That includes choosing the proper libraries and abstractions.


I think part of the resistance to testing may come from the incredible amount of dogma that comes with the "tests are good" message.

Personally, I often find that mocking everything up the wazoo to make perfect unit tests is more work than it's worth. I don't write tests first most of the time either.

Nonetheless, I consider myself a strong proponent of testing, and I always encourage people to test their code. In fact, I've started automated test suites at my two past jobs.

I've written about testing in a number of blog posts, and in particular about the conflation of TDD with unit testing.

I think the message about testing should be start with a few simple messages. First, even a few tests are infinitely more useful than none. Second, write tests, before or after, it doesn't matter, but you must write tests. Third, tests are code, so be prepared to maintain and refactor them.

Just to clarify, when I say there's too much dogma, I don't mean you (chromatic) are too dogmatic. Testing has become much too associated with things like agile development (its own annoying dogma) and ruby wankers.

Tests are good and useful all on their own, and they come in more forms than just unit tests.

Interesting article overall, except for one point. I think you've misunderstood what Jay Fields was talking about that blog post. I don't see anything in that post saying that tests are hard to maintain in general, and because of that, shouldn't be done. What he's referring to is that some types of tests consume a large amount of time to maintain, and give very little value. So if you want to do them, you should consider it carefully. He is a strong believer in unit tests and testing, and talks about various ways of doing it all the time on his blog. In fact, his very next article talks about ways to make testing stubs easier.

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide



About this Entry

This page contains a single entry by chromatic published on March 19, 2010 11:54 AM.

Why Modules and Plumbing was the previous entry in this blog.

The Difficulties of Unwritten Community Standards is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Powered by the Perl programming language

what is programming?