Why Test::Harness Should Not Enable Global Warnings


Test::Tutorial needed some attention. I've referred to it from various places in the decade-plus of its existence, but when I mentioned it most recently in the new edition of the Modern Perl book, I looked at it again.

The tutorial holds up pretty well for its age, but it didn't mention done_testing, which is a great solution to the "Should my test file declare a plan?" In short, you have three options for figuring out if your test program ran to completion: count all of the tests you expect and predeclare that number, skip it all and hope, or call done_testing at the normal exit point of your program. If done_testing runs, the test run succeeds. If done_testing doesn't run, an exception or other abnormal exit interrupted the test run, and the test run fails.

Along the way, I took the opportunity to clean up a few other things. In particular, I removed the -w flag from the hash-bang lines at the start of the test programs. (A flag enabling global behavior? In a test file? In 2011? More than a decade after the introduction of the perfectly cromulent lexical warnings implementation? DELETE!)

Schwern objected, as is his right, giving four reasons Perl tests should run with -w. In particular, he believes that it is the responsibility of the consumer of a dependency to handle warnings in dependencies.

I can't agree.

Consider the possible cases where a dependency may produce a warning.

package MayWarn
    sub no_warning
        my $foo;
        return "<$foo>";

        use warnings;
        sub lexical_warning
            my $foo;
            return "<$foo>";
            no warnings 'uninitialized';

            sub disabled_warning
                my $foo;
                return "<$foo>";

It's pretty clear that interpolating an undefined value into a string will cause a "Use of uninitialized value" warning in two circumstances: where global warnings are enabled and where lexical warnings are enabled.

No Lexical Warnings

Compare calling no_warning with and without -w:

$ perl -Ilib -MMayWarn -e 'MayWarn::no_warning()'

$ perl -Ilib -MMayWarn -w -e 'MayWarn::no_warning()'
Use of uninitialized value $foo in concatenation (.) or string at lib/MayWarn.pm line 6.

In this case, -w affects the behavior of code written without regard for lexical warnings.

Lexical Warnings Enabled

Compare calling lexical_warning with and without the -w flag:

$ perl -Ilib -MMayWarn -e 'MayWarn::lexical_warning()'
Use of uninitialized value $foo in concatenation (.) or string at lib/MayWarn.pm line 14.
$ perl -Ilib -MMayWarn -w -e 'MayWarn::lexical_warning()'
Use of uninitialized value $foo in concatenation (.) or string at lib/MayWarn.pm line 14.

The -w flag is irrelevant to the reporting of warnings, because lexical warnings are always in effect.

Lexical Warnings Enabled, Uninitialized Warnings Disabled

Compare calling disabled_warning with and without the -w flag:

$ perl -Ilib -MMayWarn -e 'MayWarn::disabled_warning()'

$ perl -Ilib -MMayWarn -w -e 'MayWarn::disabled_warning()'

The -w flag is again irrelevant, because lexical behavior overrides global behavior.

-w in Test Suites

I see the point that the rightest thing to do is to track down every CPAN module which isn't warnings-clean and try to convince the authors to Do The Right Thing, but isn't blindly applying global behavior in the hopes of finding and convincing other people to fixing bugs the same argument so many people had against UNIVERSAL::isa? (Just consider how much buggy code is still out there, years into that argument.)

Furthermore, modules which use lexical warnings properly (code written to at least Perl 5.6 standards—a March 2000 level of modernity—receive no benefit from -w. That flag is irrelevant.

Code written to the previous millennium's standards does offer the possibility of benefit from -w applied...

... but I'm not going to be the one arguing that it's sensible for Test::Harness to magically add flags I didn't ask for, and that that obligates other authors to modify their code.

Maybe I'm wrong, and maybe the desire to improve the CPAN trumps my desire to be able to reason about what my code does (and to, you know, use features lexically as I see fit), but it seems to me that the correct place to apply the big hammer of All CPAN Code Should be Free of Warnings, Globally (But Thou Canst Use Lexical Warnings, You Philistine) should be in the CPAN testers, which can test things in and of themselves.

Even so, I can't justify blindly injecting global behavior into what should be protected lexical scopes. Test::Harness doesn't have enough information to decide why dependencies written without use warnings; or no warnings; lacks either pragma. Perhaps the author doesn't know about it. Perhaps the author doesn't care. Perhaps the code is free of warnings, but the author distributes the code without the pragmas enabled to save time or memory (yeah it's probably misguided, but it happens). Further, Test::Harness can't know whether any warnings represent actually dubious code, or whether they're useless. (A handful of experienced and respected and intelligent Perl 5 programmers have argued that the uninitialized value warning does more harm than good, and sometimes I agree.)

What do you think?


I agree completely with all of Schwern's reasons. And I always use -w in my tests so that I can be just as aware of issues as my users are, and to make sure it was not I who screwed something up.


chromatic wrote:

A handful of experienced and respected and intelligent Perl 5 programmers have argued that the uninitialized value warning does more harm than good, and sometimes I agree.

In my own code, whether written for myself, $job, CPAN or the open source projects in which I participate, an uninitialized value warning:

  • 100% of the time indicates a place where I haven't thought through the code completely and can make improvements;

  • 80% of the time indicates a place where something can actually go wrong.

In a work environment where few other people write tests, an uninitialized value warning indicates a place where something can go wrong more than 95% of the time.

So, not surprisingly I try to stomp out all uninitialized value warnings -- and I usually find that the effort needed to do so is not great. Do it often enough, and after a while you don't have to think about it even in your first-draft code.

Thank you very much.

Most of the time it's useful in my code too, but I run into cases where having to initialize a variable to please warnings is busy work--autovivification handles the essential logic just fine.

Perl isn't very often a language where you have to work around the compiler, so the experience of having to do so is jarring.

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide



About this Entry

This page contains a single entry by chromatic published on October 5, 2011 9:58 AM.

Tolstoy, Emerson, Style, and Maintainability was the previous entry in this blog.

The JFDI Theory of Language Adoption is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Powered by the Perl programming language

what is programming?