Why I Use Perl: Testing

| 8 Comments

Of all the languages I've used for pay and for hobby, none compare to Perl in terms of testing culture and ecosystem.

Sure, with a few seconds and your favorite search engine anyone can find countless examples of awful code written by people who had no concern for writing good code. (That's not a language problem.) Sure, you can find countless examples of Perl code written to the standards of 1992 with little regard for documentation or formatting or robustness or even the minimum effort at basic procedural programming. (That's not a language problem.)

Shameless plug: I wrote a book called Modern Perl. You can buy Modern Perl: the book in paperback from Amazon (and other booksellers) or buy Modern Perl: the book in Kindle format—or read it online or download it as PDF or ePub for free. Furthermore, if you'd like to talk about how to improve the testing of your product or project, I am available for consulting.

One of the reasons we can talk about such things as Modern Perl is due to the quality of Perl's testing culture. I had a seat in the front row for almost all of the Perl testing and quality revolution, starting in 2000.

Actually it starts in 1987. If you find and download a Perl 1.0.x tarball, you'll see that it includes a tiny test harness and a series of language tests. This predates the notion of Test-Driven Development. It predates even Test-First development. (As far as I can tell, it even predates the invention of SUnit, the Smalltalk test framework that inspired xUnit, arguably the most popular style of testing in most languages.)

Update: As Larry himself said in a 1997 Larry Wall interview with Linux Journal:

You can restructure all your code into modules and unit test it in a jiffy because the Perl interpreter is so handy to invoke.

In 2000 and 2001, Perl 5 started taking testing more seriously. Even though Perl 5 has no formal specification outside of "Whatever the implementation does, as long as the documentation agrees," a group calling itself Perl QA took up the banner of developing tests, a testing system, and a testing culture to help Perl grow and evolve safely and on purpose through the next phase of its life.

As part of that process, Michael Schwern and I developed a library called Test::Builder to unify the internals of multiple test libraries and to allow future test libraries to share the same backend.

It's been wildly successful.

It's been so successful that you can download from the CPAN today hundreds of testing libraries which are all composable and play together nicely in the same process in the same file. They all work together with the standardized test harness and reporting tools because the Perl world does agree on formal standards. See TAP.

(You don't even have to use Perl to take advantage of TAP. I've written TAP emitters and Test::Builder libraries in multiple languages.)

That's one area of success. Another area of success is the adoption of testing and testing tools by people who don't write testing tools. (Of course people in Perl QA would use these tools, but if they never reach anyone else, what's the point?)

After Schwern and I made Test::Builder, I started to work on the test coverage for Perl 5.8 and its core library. So did other people. The number of tests of the core language and its libraries quadrupled. So did the quality of those tests, as the adoption of newer, better test libraries improved. So did the quality of those tests and those libraries as we gained experience writing good tests and understanding how to write better tests.

The quality and test coverage of CPAN and deployed Perl applications improved, too.

As I wrote in Why I Use Perl: Reliability, it's reasonable to expect that you can install a new version of Perl 5, run all of the tests for CPAN dependencies, run all of the tests for your application, and everything will just work. This isn't magic. It's science.

As part of the process of developing Perl 5, a few people run automated test runs of the CPAN against commits to the new version of Perl 5 in progress. Read that sentence again carefully. Automated processes can tell you when a commit to Perl 5 in progress causes a test to fail on the CPAN—not merely one of Perl 5's core language tests or a test in the Perl 5 core library, but a test in a CPAN distribution. An automated process will notify the maintainer of that CPAN distribution as well as the developers of Perl 5 with a link to the offending commit.

The collective test suite of the CPAN (as of this writing, 108889 modules in 25473 distributions, for a collection of millions of tests) is the continuous integration test suite of the Perl 5 language itself.

Similarly, a larger army of automated test runs reports the test results of new CPAN uploads against a huge array of platforms and Perl 5 versions. This is CPAN Testers. Within a few minutes of uploading a new distribution to the CPAN, you may get back a test result. Within a couple of days, you will have plenty of test results.

The infrastructure is there. The will to quality is there. The history of encouragement with code and documentation and tools and community expectation is there. The onus is on people writing Perl to take advantage of the testing ecosystem to write the right code.

(Did I mention that other great tools exist to test your code coverage, to test the coverage of your documentation, and even to add tests for coding standard and awkward semantic violations?)

I've written code in a lot of languages. I've debugged code in all of those languages. I've used TDD in most of those languages (excluding Postscript and minicomputer BASIC). While I've seen a focus on good testing in many of those language communities, Perl's the only language community that I see that takes testing really seriously.

(Addendum: the existing Perl Testing book is still decent, but I have this recurring notion of writing a new one. If I started a Kickstarter project to gauge interest, would you pay for early access to electronic versions of the book in progress?)

8 Comments

I'm eager enough to see an update to the testing book that I would back such a project.

Also, TAP is indeed awesome. I once wrote a simple dumb TAP test harness for REBOL, just because I could.

I would also back an update to the testing book, if only to not have it feature a new function (test_pass) that lasted 1 week.

If I had a mind control laser or a time machine, I'd fix that.

FYI: Git (distributed version control system) has testsuite implemented in shell; it moved from custom output format to TAP to make use of Test::Builder stuff, i.e. using 'prove' and Smolder (though the latter fell through, IIRC).

It would be great if `smolder` regained some life. https://github.com/mpeters/smolder (CI system in perl)

I tried using the git version of TAP-emitting shell tests for a recent project, but found it didn't quite do what I wanted, ended up rolling my own (https://github.com/illusori/bash-tap) inspired by Test::Builder and Test::More.

TAP kicks ass, as do Test::Builder and Test::More, every time I have to use xUnit testing for work I find myself wishing...

The testing culture within Perl is probably one of the strongest distinguishing characteristics of the language and community.

As a module author, cpantesters is an invaluable framework that is completely unrivalled in any other language, it's hard to express just how damn useful it is when you're developing a new module.

It's like having your eyes opened to the benefits of continuous integration for the first time, but taking it to the next level, where you're testing against thousands of different deployment environments.

I would definitely support a Kickstarter for an updated version of the Perl Testing book. I was just planning to pull mine out to review a couple things this week.

Also require Test::More; would make awesome t-shirts, mugs, etc

Speaking of t-shirts, mugs, and other products, the testanything.org TAP logo would be good on them.

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide

Categories

Pages

About this Entry

This page contains a single entry by chromatic published on August 13, 2012 9:11 AM.

From ODT to POD (with practical dynamic language features) was the previous entry in this blog.

Serving Templated Directories from a Catalyst Controller Role is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.


Powered by the Perl programming language

what is programming?