Sacrificing the Future on the Past's Golden Altar


I believe that programming languages, libraries, and tools should get easier to use over time -- if we're careful about creating good abstractions and providing usable, simple APIs. We don't always get those right the first time. Sometimes we make mistakes.

I also believe that we should identify and fix mistakes as soon as possible. This goes against the prevailing Perl 5 philosophy which spends an inordinate amount of effort trying never to break existing code, even if it relied on huge mistakes and interfaces broken-as-designed.

Contrary to some reports, I am sympathetic to the idea of keeping existing code working. It's infeasable to change thousands of lines of working code just to keep up with fashions and fads. Yet it's important to weigh the benefit of simplicity and correctness for all new code written in the future against the potential of breaking existing code which may never run on an updated version.

This is especially true for some of the core library which has been part of Perl 5 since the beginning. Consider the working-but-much-maligned File::Find, which features one of the worst possible interfaces imaginable. File::Find has been part of Perl 5 since the start:

use Modern::Perl;
use Module::CoreList;

say Module::CoreList->first_version( 'File::Find' );

# output: 5

File::Find traverses a directory tree, looking for files or directories which match arbitrary criteria. You pass the find() function a subroutine reference as callback and a list of directories in which to look. For every file or directory found, the module calls your callback. A typical use looks like:

use Cwd;
use File::Find;

sub wanted { ... }

find( \&wanted, cwd() );

My favorite part of the documentation for File::Find is:

The wanted function takes no arguments but rather does its work through a collection of variables.

$File::Find::dir is the current directory name,
$_ is the current filename within that directory
$File::Find::name is the complete pathname to the file.

In the fourteen-and-a-half years since the release of File::Find, many people have questioned the design of an API which uses global variables to send data to a callback function. (Perl has allowed passing values to functions for over twenty-one years now -- since Perl 1.0 in 1987.) Global variables have been a bad idea in programming languages since just about the invention of structured programming.

Yet this wasn't fixed in 2000, when I noticed it. This won't be fixed in 2009. Why not? In 2000, the answer was "too much code depends on the existing behavior". That was the wrong answer then, and it's worse now. Nine years later, even more code depends on the existing behavior. Entrenched mistakes dig the hole ever deeper. Over time, it gets more difficult to correct problems -- not easier. By refusing to make a necessary but backwards-incompatible change, the Perl 5 developers penalized current and future users and continued to penalize existing users, all for the sake of not penalizing an unknown number of existing users no one had surveyed or counted.

Prioritizing the past over the future is a great way to ruin your language's future.

A lot of people pin their hopes on Perl 6 for a cleanup of the Perl language and its libraries. I believe the design of Perl 6 improves every part of Perl that it touches. Yet for all of the care that has gone into Perl 6 and will continue to go into Perl 6 through its twenty-year lifespan, the same subtle temptation will plague every contributor. Unless its designers, developers, maintainers, and contributors practice the difficult discipline of relentlessly removing misfeatures in favor of cleaner, saner, smarter, and easier to use and to understand replacements, Perl 6 will eventually go the way of Perl 5: a shining gem of a language buried under accreted muck.

I hope to help wipe away some of the muck of Perl 5 -- but I want to prevent that muck from accumulating in Perl 6.


An excellent point backed by an excellent example. File::Find is so poorly designed that one sees lots of replacements out there and tons of code goes through a tortured algorithm to find files, just to avoid File::Find. This is in fact something that needs to be fixed immediately to improve the language - its just that simple.

Let's be specific - what would you propose as the fix for File::Find? Changing it's API? Replacing it by a saner alternative in the core library?

Maybe I'm crazy, but why not just add a new function with a saner interface and deprecate the old one?

Does anyone have a viable alternatives to File::Find in the short term? I've used it and love it, and have yet to find anything (short writing my own tree-traversing code) that will accomplish the same thing.

Also, as a newbie to Perl, and not knowing better, I find myself declaring global variables that are lexically scoped (using "my"). But I keeping reading (here among other places) that listing my $var; or my $var = ''; at the beginning of a script is not the way to do it. Does anyone know a good place to look for the correct way to do this? How do you share variables between functions in the same scope, in the main scope, without doing this? Thanks!

I wrote a response to this post on my journal. I mention there File-Find-Object, which I actively maintain and solves all the major drawbacks of File::Find that I'm aware of.

Also see this page on the Perl 5 Wiki.

Seems like this problem can be solved similarly to the File::Spec one. File::Spec is also fairly ugly, Path::Class came along, which builds on top and has a nicer interface. Many people use it, and require it in their modules. If we could leave File::Spec in place but replace its docs with "go use Path::Class", it could be phased out eventually and no-one would notice.

Do the same with File::Find, have a much better replacement, get it installed everywhere by dint of modules requiring it. Hide File::Finds docs.

Backcompat can't be maintained forever, mistakes have to be fixed, or as you said, we just get a bigger mess. The mantra should become "if your code works and you don't have time to port it to newer perls, don't upgrade the perl on that box, leave it be".

(Or those crazy people could maintain their own copy of F::F when they upgrade and its gone from Core)

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide



About this Entry

This page contains a single entry by chromatic published on February 3, 2009 1:20 PM.

Eliminating Messy Code with Sentences and Paragraphs was the previous entry in this blog.

Stop Preventing the Future! is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Powered by the Perl programming language

what is programming?