Replacing the Standard Library with Distributions


Languages evolve.

Some people want to put closures in Java. (Set aside the fact that only 10% of the people I've seen talk about closures in Java know the difference between first-class functions and closures.)

C++ will eventually get a new specification.

COBOL has OO extensions.

Despite the fact that Microsoft can't seem to support C99 in Visual Studio (hey, give them a break -- they're still working on supporting PNG and CSS 1.x, two standards even older than C99), C# gets new features.

You don't have to use the new versions. You don't have to use the new features of the new versions. You don't have to like the new features of the new versions, and you don't even have to know they exist.

You have to delve deeply into delusional solipsism to deny that they exist... and that's merely language features. Consider all of the changes in libraries available for those languages.

In The Replaceable Batteries of Your Standard Library, I argued that change is even more constant when you consider the available libraries for a language. This is true of standard libraries blessed by the implementors (try using gets() in a modern C project and see the reactions you get) as well as libraries made popular by common use (remember Class::DBI?).

Change happens.

Language designers and developers and maintainers and patchers and porters and releasers ought to consider how to deal with the inevitability of change.

Should a new Perl project use CGI today? Plack seems like a better option. Should a new Perl project start with Perl 5's basic OO or use Moose? Is there any excuse to recommend the use of File::Find, when the basic API requires the use of features that 90% of Java programmers (in my unscientific and unrepresentative experience) do not understand?

The best argument for using CGI and bare-bones Perl OO and File::Find is that they're available on whichever version of Perl 5 you're likely to have installed (even if that's Perl 5.005, which has been obsolete for the whole of the 21st century and even if they have bugs which have been fixed in the past several years). The second best argument for using them is that there's plenty of example code which demonstrates how to use them (though if you write Perl 5 code based on examples you can find online... ouch).

The third best reason is that, once something is in a language's core library, only an act of the language designer can remove it. In theory, you'll get a lengthy period of deprecation warnings (if you pay attention to that sort of thing, but convince me that the people not using Moose and sticking with Perl 5.6.1 know how to get deprecation warnings from Perl 5). In practice, you're in trouble anyway, unless your language has a specification or a comprehensive test suite and you read it to figure out if specific behavior on which your code relies will always be present in the future.

In other words, change happens.

How can a language designer or release manager or maintainer or porter walk the thin line between providing a bare-bones language with few libraries and cramming everything into a core bursting with subtly-different alternatives and growing less relevant to real-world adept uses because of too-early blessings of approaches which time has proven inopportune for the common uses?

I've argued before that the right approach is a small core which contains only those libraries absolutely necessary to bootstrap an intelligent library loading system. This works for operating systems such as Gentoo GNU/Linux, Debian in its flavors, and many of the BSDs. Parrot is exploring a similar approach.

The Perl 5 core could do very well to include only a couple of libraries necessary to build and install Perl and to download (or otherwise build from a download or checkout) a bare-bones CPAN client. Perl 6 may do something very much like this.

That leaves one question unaddressed. What happens to those unfortunate novices who download the core distribution themselves and find out that they can't do much with it? What happens to people with cruel, cruel sysadmins who refuse to install anything other than the core?

The solution comes from realizing that the people who are best at maintaining and porting and releasing a new version of a language and its core are not necessarily the same people who are best at discovering what the language's community considers most useful, most usable, or most appropriate.

You can already see the solution in something like Strawberry Perl, which unabashedly includes a compiler and is on its way to including an IDE and a GUI toolkit -- not because you can't use Perl 5 effectively without any of those, but because Perl 5 is much more powerful and much more useful with them.

That line of reasoning is very similar to the one which includes any given library in the standard library.

Why not embrace it? Let distributions come up with their own lifecycles and deprecation and suppport policies. Let them identify which libraries people should have available by default for the next eighteen to twenty-four months. Let them perform the work of integration and testing and bug reporting and working with upstream authors.

Upstream would always remain the same. If you can access CPAN, you can ignore distributions or install your own additional packages or build your own distribution. (Strawberry makes its own tools available.)

A similar process works well for operating systems -- and, arguably, operating system users are less savvy than programmers. Many distributions already package Perl 5 this way.

Could you live with such a change?


Having a small, stable, core has worked well for the CGI::Application ecosystem.

A number of plugins are maintained separately from the core, and some Perl distributions package together useful collections. Titanium is the best known of these, but there are others, such as CGI::Application::Structured.

I think a similar approach could work for the Perl core, as well. Heck, one of the distributions that someone could choose to package could even be all the Perl modules which used to be the core. :)

I think it's an idea who's time has come.

Back in September the Python folks had quite a long discussion about
issues with their standard library (it began here:

Jesse N. noted that some say "The running joke is that the stdlib is
where modules go to die". One of the goals of Python 3 was to clean up
their std lib. I would guess it's a difficult task mostly just due to
the nature of large standard libs: code gets in, people begin relying
on it, there's pressure to not break backward compatibility, then core
devs have limited time to keep everything maintained.

I didn't follow the discussion to its end, so I don't know if they
came up with a solution. I think maybe they plan on eventually
physically separating out the stdlib in version control, but still
distribute it as batteries included and users would be none the wiser.

Another community dealing with the opposite problem right now is the
Scheme community. From what I can tell (since I don't know Scheme),
they've languished a long time with a completely balkanized community.
Many and varied implementations, not too much interoperability. It's
only now with the beginnings of their recent R7RS effort that it looks
like they should get things like a common module system so modules
could actually be used across different Scheme implementations
(!). More about R7RS here:

I think one of Perl's greatest strengths is the ease with which usage
of one "standard" module can be replaced with another if the other
turns out to be better than the former. It's short-term pain for
developers because there's more research involved in choosing which
modules to use, but long-term it allows the common usage of the
language to evolve, getting better over time in a way that it couldn't
do otherwise.

So, yeah, I'm all for having distributions replace the standard lib
(for Perl 6). That would mean I could choose the most modern "best
practice" one rather than the "has everything to be backwards
compatible" one. :)

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide



About this Entry

This page contains a single entry by chromatic published on December 10, 2009 5:47 PM.

The Replaceable Batteries of Your Standard Library was the previous entry in this blog.

Perl and the Multiversion Grammar is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Powered by the Perl programming language

what is programming?