Why is Funding Perl Core Development So Difficult?


PHP has Zend. Python has Daddy Googlebucks. Java has... let's call it the 1% of programming languages. C# has Microsoft. JavaScript has everybody who's ever written a web browser (except possibly the W3C).

I haven't used Windows for work purposes in ages, but I'm glad ActiveState exists—Perl 5 is much better on Windows for their assistance.

I'm also glad to see Perl Foundation sponsors, especially the ones who've contributed to the Fixing Perl 5 Core Bugs Grant.

Yet I wonder why it's so amazingly difficult to get full-time funding for new Perl 5 core development. (Again, this is not to complain about funding for Nick Clark or Dave Mitchell, because their work is important and valuable and everyone who's contributed deserves gratitude.) I can't decide what I think, but I have some possibilities:

  • Perhaps Perl is just too effective. It's a tool you apply when you need to do something quick or dirty or now, and after that it just works and you forget about it. (There's a lot of code like this in the world. Then again, a lot of small businesses still run on Access and other 4GLs.)
  • Perhaps the dominant perception of Perl 5 is a life support system for the CPAN. (Much of the evolution of Perl happens on the CPAN to be sure, and so volunteer effort goes there.)
  • Certainly it's too difficult to hack on the Perl 5 core, so the available contributor pool is not in what anyone might legitimately call a state of growth.
  • Perhaps new development in Perl 5 tends to be plumbing tasks and not big, bet-the-company technical choices. (Even though my business relies completely on Perl 5 for its technology stacks, I've long realized that I'm not representative of the world as a whole.)
  • Perhaps everyone assumes someone else will take care of it. (Larry hasn't had a patron since 2000.)
  • Perhaps Perl 5 has reached minimum viable utility as it is and needs no more new features. (I have trouble believing this. I could certainly use better parallelism. I've long wanted either or both of hygenic macros and continuations. I could use multiple dispatch today. Grammars would be wonderful. Opaque objects would be grand. Who doesn't want a JIT or more speed or lower memory usage?)
  • Perhaps backwards compatibility is cannibalizing usage from current (and future) releases. (Red Hat and CentOS, please feel free to join Perl 5 in the 21st century. We're pretty sure this century will stick by now.)
  • Perhaps TPF isn't effective at courting donors. (I hesitate to bring this up, because it sounds like criticism, but that's not the intent. I do believe TPF has successfully courted some donors, which is the reason why Nick and Dave have funding right now, and I have no desire to criticize the work or abilities or interests of volunteers doing things I have no desire to do, but it does seem fair to say that I haven't seen much effort in the past five years to talk to large Perl shops to express a coherent vision for core development.)
  • Perhaps (and I find this most likely) there's no coherent vision for Perl 5 core development. Jesse Vincent's Perl 5.16 and Beyond (video link) lays out a good and effective and necessary philosophy for how to manage changes, but is there really anything to get excited about? strict and warnings by default?
  • Perl 6 didn't save Perl. (Yes, yes, Milestones in the Perl Renaissance, but "Someday this will be amazing!" becomes less amazing the longer someday takes.)

It's no one thing and it's probably a combination of several things to various degrees. I hope that someone like the amazing Mike Pall of LuaJIT might come along and demonstrate a powerful proof of concept of something exciting (better parallelism/concurrency, macros, an improved parser, a working JIT, a no-XS extension mechanism, an 80% port of the sanest parts of Perl 5 to a different virtual machine, whatever). Maybe that's Reini Urban, and maybe it's someone we don't know about yet.

It's difficult to imagine someone new jumping in to the big wad of heavily-macroed C code that's the current Perl 5 implementation and having all of the time, interest, and energy to learn what's going on as well as the luck, skill, and patience to make substantive changes without horribly breaking a dozen things elsewhere while successfully convincing p5p that the changes are worthwhile and maintainable and won't be the subject of massive imprecations and furrowed brows in two years.

Maybe I'm just a pessimist today though.

What do you think?


> Maybe that's Reini Urban

I kinda doubt that, seeing how he just recently posted on P5P that he thinks 5.6.2 is still the best Perl.

Most of these projects chromatic lists have either an engaged BDFL or some money bags with a marketing department. Perl 5 effectively has neither. We need to acquire one or the other or 5 is going to continue to appear as a boat adrift.

Perl 6 has bifurcated the effort. Good talent, steering, and effort is spent on 6 that was not spent on some other project or on the perl 5 core. Since 6 has been hanging out there for as long as it has, decades if we include topaz, it gets identified as a death march project, even if insiders don't see it that way.

Regular, incremental releases of the perl 5 core are a fantastic recent result. Massive kudos go to those who make that possible. They have made Perl 5 a vibrant platform again. These releases have turned Perl 5 back into a moving wagon. Still from my dilettante, mere user point of view, the BDFL of the Perl project has been associated with the apparent death march rather than the moving wagon. Direction of the perl 5 effort is managed by crew and not by the captain. People want to see a captain. We need to either adopt one or recover the one we already have.

I don't think that one big sponsor is what the perl community would fit.
I'm wondering if the "wikipedia way" with regular calls for donations was an option for perl. This could even end in "annual subscriptions" for individuals or companies.
I am unaware if and to what degree this already happens, but I feel that a little money could do a lot.
A rewrite of perl5 - if that is what you suggest - could be an interesting option if it is thought to be around for another 20-150 years. But (only) with solid sponsoring, a schedule and some managers to coordinate this effort.
All in all it would be very sad if perl and its philosophy disappeared from the world of programming "only" for the lack of money.

Good talent, steering, and effort is spent on 6 that was not spent on some other project or on the perl 5 core.

That is actually a myth. The overlap between the 5 and 6 communities is limited, and most people who joined the 6 effort neither came from 5 nor have much interest in it, if any.

I agree that as it stands, 5 has no editorial direction (especially none with continuity, which has resulted in the still-unfolding disaster of 5.10).

What do you mean by "the still-unfolding disaster of 5.10"? The silly feature pragma? The odd polymorphic behavior of the smart-match operator? Both? Neither? Something else?

Both of those, as well as lexical “$_”, its broken use in “given”, and the bat-shit insanity of “when” added on top of smart match. Almost every prominent feature added in 5.10 was a catastrophe all on its own, never mind their confluence.

(The exceptions are telling: “say” and “//” are good. I do not want to miss those, they feel right at home in the language. There are also much less prominent features, all of which tend to be good (which again is telling), such as a number of the additions to the pattern syntax.)

All of the failures have yet to be unravelled to any significant degree. And whether the fixes will be right is an open question. Hence “still-unfolding”.

If you cite me please right.
I said that perl5.6.2 is still the *fastest* not the *best*. The best is 5.15.5, but I bet someone will destroy it in the last minute.

My old core plans were outlined at the corehackers wiki:

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide



About this Entry

This page contains a single entry by chromatic published on November 16, 2011 3:28 PM.

Modern Perl Book Draft Period Ending Soon was the previous entry in this blog.

Promoting Perl's Features versus Benefits is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Powered by the Perl programming language

what is programming?