I've written before that software projects need sane, published deprecation policies, and I still believe that...
... but listen to a tale of two intertwined projects with a seemingly-sane and published deprecation policy that sounds great but doesn't actually work.
Parrot and Rakudo used to be much closer together. The Parrot repository had a subdirectory languages/ which held several language implementations running on Parrot. This was well and good (How did a lion get rich? It was the olden days!)—whenever Parrot underwent an API change, a simple ack of the source tree could find most places that needed to change, and running the test suites of the projects under languages/ could find the rest.
(Getting people to run the full test suite is a different story, but in that case at least guilt and a good bisection tool work post hoc magic.)
Then one day, Parrot left the Perl-centric world of perl.org and kicked
languages/ out of the nest. languages/perl6 went to
Rakudo.org and the troubles began.
You see, back in the olden days when a lion could get rich and the Perl 6 VM
was like the Perl VM but a little bit better, the way to extend Parrot was,
obviously, write a whole wad of C code. Want to add a new data type?
Write a wad of C code. Want to change how a core wad of C code works? Write a
whole wad of C code. If hacking on Perl had taught anyone anything (and I
can't believe I can write this with a straight face), it's that finding hackers
ready, willing, and able to write big wads of C code to work around or
customize or extend other big wads of C code so that someone else doesn't have
to write C code is a really easy prospect.
Anyhow, the Parrot execution and extension and embedding model has always
assumed, as does Perl, that the VM should expose a big wad of C functions
that extenders and embedders should use to manipulate big wads of C structs and
other data types defined in C.
On one hand, customization and flexibility is good. On the other hand,
people like me who know both C and Perl write in Perl when we can and C only
when we have to for several very, very good reasons.
Imagine a ventriloquist with a Parrot puppet, except he doesn't stick his
hand in the puppet to pull levers and flip switches and wiggle his fingers. He
has a robot hand, operated by a joystick, and that robot hand is half robot and
half organic and it tickles actual organs inside the puppet and the bird goes
off and does its thing. Oh, and he's operating that joystick with his toes.
Plus he's underwater, wearing a blindfold.
In other words, Rakudo has an intimate knowledge of the guts of Parrot and
does some strange and bizarre things. (If you look through the revision history
of those parts of Rakudo, you'll see I've fixed some of those strange and
bizarre things and I've perpetuated others.)
When Rakudo and other languages left the nest, a deprecation policy came about. Imagine this vaudeville act:
HLL Developer: "Hey, you changed this API!"
Parrot Developer: "You weren't supposed to use it."
HLL Developer: "No one told me that, but regardless, you broke my project."
Parrot Developer: "Oh, sorry. Well here's how to fix it."
HLL Developer: "Wish you'd told me that months ago."
Parrot Developer: "Yeah, we'll do better next time. By the
way, what in the world were you doing with that?"
HLL Developer: "No one told me to do it differently, so I
just threw together something that works."
You can see where this is going.
Thus began the great Parrot version numbering wars. The end result was that
Parrot would have two supported releases every year, one coming every six
months. The initial deprecation policy promised no backwards incompatible
changes without notification in at least one supported release.
In other words, if a project which was previously in languages/ had
moved somewhere else where a well-meaning Parrot developer didn't notice that a
simple change broke backwards compatibility, that project had the ability to
request a reversion of the change, a documentation of the change as a
compatibility change, and a delay of up to six months before making the
... depending on when the project noticed.
I've mentioned the idea of technical
friction before, but get out your calendar.
This policy didn't only protect languages from unintended changes, it
protected languages from desired changes. For example, if Rakudo wanted a new
feature in Parrot, and if the best way to add that feature in Parrot meant
breaking backwards compatibility (changing the order of search paths, for
example), both Rakudo and Parrot developers had to mark their calendars.
The deprecation policy period is now three months, for hopefully obvious reasons.
Even so, now there are at least two projects on separate time tables with
separate goals for features and improvements and deprecations. When Rakudo
wants a feature from Parrot that Parrot doesn't yet support (or Rakudo
developers believe it doesn't support fully or at all or in the precise way
Rakudo wants), Rakudo tends to toe the joystick. When Parrot makes a change
that exposes the fragility of tickling bird gullets with robot fingers, Rakudo
gets to complain. The deprecation policy followed to the letter prevents Parrot
from adding the features Rakudo demands at the same time that it prevents
Parrot from improving the features that Rakudo uses.
You might read this and think that the problem is the deprecation policy.
It's not—at least, that's not the root cause.
I hate the polite fiction calorie-free slogan that "Perl 6 is a research
project", because it's a polite way to suggest that Perl 6 is an embarrassing
pipe dream no true Perl hacker takes seriously, but implementing Perl 6 has
required a lot of research and fits and false starts. (Before you complain next
that it's taking a while, you write a performant grammar engine which
allows lexical overriding of any grammatical rule or category.) When Parrot
began, no one knew exactly how a Perl 6 VM machine should implement several
important features. That was also true in 2005. That was also true of Parrot's 1.0 release in
Making an effective Perl 6 means continually thinking and rethinking and simplifying and inventing new things.
Now tie an anchor around the feet of a live bird and pretend you're a
Granted, Parrot's also long had the goal of running other dynamic languages
efficiently and effectively. Its endgame is more than merely Perl 6 (and Perl
5). Yet a Parrot that won't run Perl 6 well has failed at its primary goal, and
subsequent goals are less interesting.
The right solution is to invent a time machine and not kick Rakudo out of
the nest. (The rightest solution is to invent a time machine and start
implementing Perl 6 on Parrot from the first day as a first-class citizen and
not a rat's nest of Perl parsing madness which had, to my count, at least two
replacements before Rakudo.)
Rakudo's developers—who had commit access to Parrot, of course, so
they could fix things if they wanted—declared that the only real solution
would be to add another layer, another project external to both Parrot and
Rakudo, which will sit in between the two and abstract away the Parrot specific
parts in the hope that someday someone will add another VM backend so that
Rakudo doesn't have to be tied to Parrot forever. In other words, throw away
all of the work that went into Parrot, start over from scratch, and hope that
this rewrite will be the one that sticks.
That went over about as well as you'd expect. (Chasing
away Parrot's developers set Rakudo back by another couple of years.)
If the notion of working around a broken deprecation policy—broken
because of too little coupling between two projects so intimately
intertwined—is to add another layer of coupling to separate those two
projects seems a little strange to you, you're not alone. By my count, that's
three projects in a driver's seat built for one. If you can figure out which
project is upstream and which project is downstream and how to debug with
reasonable each which change where broke which test and who gets to fix it,
you're a far, far better developer than I am.