Running Named Perl Tests from Prove

| 1 Comment

Organizing Perl Test Files showed the basic framework I use for individual test files written to work with Perl's testing tools. While I gladly take advantage of frameworks such as Test::Class and Test::Routine, sometimes I need something a little simpler.

The discipline of the organization method I explained in the previous article offers the benefit of simplicity and some discoverability. It also allows my team to run only a portion of the test suite as needed.

One of our products has a web interface. We have quite a few tests for this, but because they run through the whole web stack, they're quite a bit slower than tests for the business model API directly. (We want to make sure the web site always works, so we have some exhaustive tests.)

We've divided many of these tests up into discrete files based on controllers and subsets of controllers. The administrative section has a few test files. The data sharing section has a few test files. The public section has a few test files.

Some of these tests take a while to run—multiple seconds. If you're working on a bug or a feature in one specific action, waiting more than a couple of seconds for test results is way too long.

That's where the named functions I use to group related tests come in. Assume you have a test file testing the admin features, and one of those named functions is test_admin_console_list_expired_users(). You could edit the main() function to comment out all of the other tests. Alternately, add a simple code block to main():

sub main {
    my @args  = @_;

    if (@args) {
        for my $name (@args) {
            die "No test method test_$name\n"
                unless my $func = __PACKAGE__->can( 'test_' . $name );

        return 0;

    # ... run all tests here

    return 0;

... which will interpret any arguments passed to this test file as names of test functions to run. To run only the test function test_admin_console_list_expired_users(), use the prove command line:

$ prove -l t/web/admin_console.t :: admin_console_list_expired_users

The double-colon tells prove to stop looking for its own arguments and to pass the following arguments to the test file. With that invocation, only the requested test will run.

For this strategy to work, your test functions must be independent within your test files. The data the expired users list test needs to run must already be in place, untouched by other tests, or the test will fail. That's good test discipline anyway, though.

This strategy only saves us several seconds every time we use it, but saving several seconds from asking the question "Does this work?" to getting the answer is a huge benefit in the moment.

There may be easier ways to handle this, but so far this has met a real need without forcing us to divide our tests into even finer granularity (more files to manage and remember) but surely allowing us to run tests at the level we need most. For any test file which takes longer than two or three seconds to run in parallel, this has been a huge benefit.

1 Comment

I was thinking the same thing when you wrote part 1, but couldn't invoke the function by string name. __PACKAGE__->can was somewhat more complexity than I was prepared to devote to it at the time. ( ie, I didn't think of that)

The only problem is having this over and over in test files: "Boilerplate Considered Harmful". How about moving this into Test::Simple or Test::More, or a module of its own, Test::Some.

I would suggest exporting a single routine, run_some(), which contains the innards of the if() block. Then all a user needs do is to begin main with

     return &run_some if @_;

or the more verbose but digestable

     my @args = @_;
     return run_some( @args ) if @args;

Of course, that means main() is a boring list of tests to run. Test::Some could supply that main() if it was requested to in the use line. The user would only need to invoke main(), passing an arrayref of the test file names, in the order they should be run, followed by \@ARGV.

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide



About this Entry

This page contains a single entry by chromatic published on May 12, 2013 6:00 AM.

Organizing Perl Test Files was the previous entry in this blog.

Including People is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Powered by the Perl programming language

what is programming?