A couple of weeks ago I had to revise the data model in a client application. (Rant: when did the word "refactoring" get watered down into "changing code"? Refactoring used to mean changing the design of code without changing its behavior or external interface. If tests break, you did something wrong.) Unfortunately, this data model was at the core of one important piece of the system, and a lot of tests broke.
This was good, in the sense that we had tests for the entire feature and that they gave us a lot of coverage that we had it right.
This was bad, in that digging through a lot of failing test output is unpleasant work where you fix one thing and hope it fixes a lot of other things.
I've been working with Ovid again on this project, and one of the first
things he brought up as a technical matter is the use of Test::Most over plain Test::More and the grab
bag of other test modules that eventually creep in. Sure, he wrote
Test::Most and has a lot of Vim macros and development habits to
make his work easier, so it's not a burden to switch the code over
I'd never taken to
Test::Most, mostly because I never took the
time to use it seriously.
The other day, when facing a huge pile of failing tests I needed to make
pass again, I decided to try it for one reason:
you enable that feature of
Test::Most, the first failing test will
abort the test run. The only failure output you get is that first failing
I normally don't want this, because I want the whole gory details of my failure if I break something, but with the stepwise refinement necessary to finish my data model changes, it was just what I needed.
While I was editing the test files, I changed
use Test::Most;. That's easy enough. I didn't want to enable
BAIL_ON_FAIL unilaterally because I'm not sure I want its behavior
all the time. That's fine; you can run your tests with the environment variable
$ BAIL_ON_FAIL=1 prove -l t/web/ui_tests/some_feature.t
The responsibility for enabling this behavior is yours; it's not internal to the test file.
What if you take that one step further? What if you don't necessarily want
Test::Most inside your test file yet? You can still get the
BAIL_ON_FAIL behavior if you load
$ BAIL_ON_FAIL=1 prove -MTest::Most t/web/ui_tests/some_feature.t
Even though the test file itself has no knowledge of
Test::Most, you still get its ability to bail out of the test file
at the first failure. (You do currently get a warning about a plan mismatch,
but for me that's easy enough to overlook, as the rest of the output is much
easier to use to diagnose failures.)