Plack::Test and Great Test:: Interfaces

| 4 Comments

Plack is great. Most of it is wonderful. I'm less enamored with the interface of Plack::Test, however.

In writing the Little Plack Book, I spent a couple of days writing tests for Plack and applications at the PSGI level. Plack::Test occupies a strange level in the ecosystem. It's incredibly useful for what it does in selecting between a no-HTTP backend or any other PSGI-compatible handler, and it offers some degree of abstraction for making requests and getting results, but it's far too low level to write tests for complex applications.

When writing tests for a small Dancer application, I spent more time getting the Dancer environment set up well for testing than I did writing the tests. If I'd used Dancer::Test, I suspect I'd have made more progress more quickly.

I've had similar experiences with Catalyst and Catalyst::Test.

None of this surprises me—Catalyst and Dancer are one layer up the stack from Plack. Most of the interesting things you can test about a Catalyst or Dancer or Mojolicious or whatever application are properties expressed at the framework and application layers, not the plumbing layer of Plack.

Plack::Test seems best for testing middleware and other components which occupy the layer between PSGI and applications. Even so, something about its interface kept bothering me as I wrote tests and prose about the tests:

test_psgi $app, sub
{
    my $cb  = shift;

    my $res = $cb->( GET '/?have=tea;want=tea' );
    ok $res->is_success, 'Request should succeed when values match';
    is $res->decoded_content, 'ok',
        '... with descriptive success message';

    ...
};

The pattern of a special test function which takes a block of code is semi-common in the Test::* world; you can see it in Test::Exception and a lesser extent in Test::Fatal. The best example I've seen of this is Test::Routine, which uses this block syntax to help organize tests into named groups. A disciplined tester can use this to great effect to clarify what might otherwise become a big ball of muddy tests.

I like that Plack::Test does the hard work of redirecting requests to the app I want to test on whichever backend I want, so that I don't have to worry about setting up a testing deployment target. That part's great. The confusing part is:

my $cb  = shift;
my $res = $cb->( GET '/some_url' );

test_psgi takes a PSGI application and an anonymous function as its parameters, then invokes the anonymous function, passing a callback bound tothe context of the application. Inside the anonymous function (the block, not the callback), you invoke the callback and pass an HTTP::Request object (constructed manually or with a helper such as HTTP::Request::Common) and receive an HTTP::Response object.

Put that way, it's a lot more confusing than it is, if you're comfortable with the idea of first-class functions and closures and semi-fluent interfaces in Perl 5.

Even so, my $res = $cb->( $req ) just looks weird to me. It sticks out. It's visually different from all of the rest of the code. Everything else outside the test is the semi-fluent interface of anonymous subs or boring old method calls on objects.

In a discussion with Zbigniew Łukasiak, I suggested that I'd want an interface more like:

use Plack::Test;

plack_test 'Test description here' => sub
{
    my $app = shift;
    my $res = $app->get( '/', { have => 'tea', want => 'tea' } );

    ...
};

You can see the influence of Test::Routine. I don't know exactly how $app gets into the block, but this gives labeled subtests and the concomitant organization, it obviates the need to create HTTP::Request and HTTP::Response objects (or their Plack:: equivalents) manually, and everything in the block uses visually similar mechanisms.

The only hairy part is figuring out how to connect plack_test to the app while not multiplying hidden global variables or disallowing customization and decoration with other middleware. Compatibility with Plack::Builder is important, but I don't especially want to pass in $app myself manually.

Besides, to me the only obvious benefits over Test::WWW::Mechanize::PSGI are the subtest groupings. Mech has a huge advantage of providing many useful test assertion methods which grovel inside responses so I don't have to.

Maybe this is less my discomfort with one part of the useful-at-the-appropriate-level Plack::Test and more a plea to distinguish more clearly between various elements of Test::* modules, as several distinct categories of behavior exist:

  • Setting up a testing environment
  • Organizing test assertions
  • Providing test assertions

We did a huge amount of work a long time ago with Test::Builder making sure that everything which wanted to provide test assertions could interact nicely, and we succeeded. Maybe it's time to consider ways to enable that composition at other layers of the Test::* stack.

4 Comments

It's incredibly useful for what it does in selecting between a no-HTTP backend or any other PSGI-compatible handler, and it offers some degree of abstraction for making requests and getting results, but it's far too low level to write tests for complex applications. (...) Plack::Test seems best for testing middleware and other components which occupy the layer between PSGI and applications.

It is correct that Plack::Test is targeted to write unit tests for PSGI app compatibility as well as writing PSGI middleware. And for higher level application testing, you're suggested to use interfaces like Test::WWW::Mechanize(::PSGI) as you pointed out in the rest.

Even so, my $res = $cb->( $req ) just looks weird to me.

It looks as weird as my $res = $app->($env); looks in the PSGI - the point is, or at least originally was, the Plack::test interface should be wrapped and presented as the Catalyst::Test/Dancer::Test interface as an application developers - such that app developers don't need to worry about which PSGI backend or handler it is using.

The other reason I chose the callback interface was to take the benefit of Test::TCP module, that expects the callbacks to be executed by forking the server and client code apart. You can argue that is an implementation detail, but at that time we wrote Plack::Test (in late 2009) that was the only (easy) way to achieve that.

That said, newer version of Test::TCP supports the OO interface that doesn't require two callbacks - only once for the server code which can be just a wrapper for the PSGI $app callback, and client caller code can be written in a synchronous looking code. There's even a ticket for that to put that back to Plack::Test, which I closed for some reason I don't remember - but it might be worth trying it again.

Zbigniew and I talked about the correspondence between the testing callback and the PSGI callback. Plack::Test's callback is very Plackish, and that's an appropriate design choice. It does the job and well, and I understand why you chose it.

Mixing of different interface styles in client code just seemed jarring. Maybe it was me trying to formulate the idea that Test::* modules provide different parts of a testing stack.

Yeah, there was a reason that it is called Plack::Test, not Test::Plack, I guess :)

I like this approach and would welcome to use Plack::Test::Agent from CPAN if the "This is an experimental module and its interface may change" is removed.

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

affiliated with ModernPerl.net

Categories

Pages

About this Entry

This page contains a single entry by chromatic published on September 21, 2011 10:48 AM.

Plack::Test Backend Selection was the previous entry in this blog.

Introducing Plack::Test::Agent is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.


Sponsored by Blender Recipe Reviews and the Trendshare how to invest guide

Powered by the Perl programming language

what is programming?