Temporary Directory Handling in Tests

| 2 Comments

Per my experiments with parallelism in Perl test suites, I've adopted several patterns. One such pattern allows me to manipulate the filesystem in a parallel-friendly way.

A lot of my code handles batch processing: fetch some data, sort it into various logical buckets, manipulate the contents of each bucket, then produce some sort of output for everything that made it that far. While most of these steps only manipulate active data in the queue, some steps require me to read and write files in the filesystem—see Pod::PseudoPod::Book (soon to be on the CPAN) for example.

As with any parallelism, multiple units of execution contending over the same single shared resource is an exercise in conflict, or at least complicated locking.

For simple needs, File::Tempdir is great. You can create a temporary directory with a lifespan tied to the object representing it. When that object gets destroyed, its destructor removes the temporary directory.

I needed something more. I wrote the very silly, very simple Tempdir solely for one project's test suite:

package Tempdir;

use Cwd;
use autodie;
use File::Temp;
use File::Path;

use base 'File::Temp::Dir';

sub new
{
    my ($class, %options)       = @_;
    my $self                    = File::Temp->newdir;
    @{ $self }{ keys %options } = values %options;
    $self->{original_dir}       = cwd();

    chdir $self->dirname;
    File::Path::make_path( @{ $self->{mkdirs} } );

    bless $self, $class;
}

sub write_file
{
    my ($self, $name, $contents) = @_;
    open my $outfh, '>', $name;
    print {$outfh} $contents;
}

sub DESTROY
{
    my $self = shift;
    chdir delete $self->{original_dir};
    $self->SUPER::DESTROY( @_ );
}

1;

Like File::Tempdir, creating a new Tempdir object creates a temporary directory. In addition, it saves the current working directory and chdirs to the new temporary directory. Because I'm careful to use only relative paths within my code (business requirement: prefer running multiple related instances of a project on a single machine to separate virtual machines), as long as the relative necessary files and directories are present, everything continues to work correctly. (Also because this temporary directory manipulation happens at runtime, the test file's connection to the work queue is already in place, so chdir works just fine.)

If you provide the constructor a mkdirs key with an array reference as its values, the object will create, relative to the temporary directory, additional subdirectories of arbitrary depth. I also added a very simple convenience feature to write a file. I haven't needed more than this yet:

# create the storage directories for topics/2
my $tempdir = Tempdir->new(mkdirs => [ 'sites/Bravo', 'sites/Bravo/css' ]);

...

$tempdir->write_file( $css->filepath, $css->contents );

When $tempdir goes out of scope, all of these files and directories go away. Even if I were to run a hundred instances of the same test file simultaneously, they would all run successfully because they do not interfere with each other.

Though I chose an OO interface for this behavior, I prefer a higher-order interface in some ways. I'd like to be able to write:

within_tempdir mkdirs => [qw( some list of directories )]
{
    # do something
    ...
};

... but I haven't convinced myself quite yet that it's an improvement. Certainly it has the potential to be more correct, as nested lexical scoping has a better chance of applying and unapplying chdir calls in the correct order (it behaves more like properly associated pushd/popd calls in bash), but Perl 5's limited abilities for parameterization of these thunks is clunkier than it ought to be. I could experiment with an interface where you specify parameters to import which produces and exports a partially applied function, but the OO version is good enough for me for now and continues to stay out of my way.

2 Comments

chromatic wrote:

For simple needs, File::Tempdir is great.

Did you mean this: File::Tempdir? Or this: File-Temp?

File::Tempdir, your first link.

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

affiliated with ModernPerl.net

Categories

Pages

About this Entry

This page contains a single entry by chromatic published on November 29, 2011 10:07 AM.

Solving Problems or Absorbing Design Patterns was the previous entry in this blog.

Controlling Test Parallelism with prove is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.


Sponsored by Blender Recipe Reviews and the Trendshare how to invest guide

Powered by the Perl programming language

what is programming?