The Perl world often speaks of the DarkPAN. This is the CPAN's big brother. Where the CPAN contains millions of lines of freely-available, reusable, and tested Perl code, no one knows how big the DarkPAN is, what it contains, or if it's tested. Where there are CPAN testers, CPAN metrics, CPAN searching, comprehensive CPAN history, and CPAN code search and cross referencing, none of those features are available for the DarkPAN.
In short, we don't know where the DarkPAN is. We don't know what it contains. We don't know which versions of Perl it uses, which CPAN modules it uses, which XS functions it uses, which Perl idioms it uses, and which bugs it relies on going unfixed. It's a big wad of unknown that we suspect exists but can't measure.
The Perl 5 Porters have a fabulous resource named Andreas J. König, not coincidentally the maintainer of the CPAN module. Andreas often posts Bleadperl Breaks CPAN reports, where he identifies specific changes to Perl under development which have caused test failures in CPAN modules. At that point, the question is whether the change to bleadperl is wrong, or whether the module did something wrong. One or the other gets fixed.
That doesn't work for the DarkPAN. CPAN testing is automatable (proof: it's automated, and has been for years). DarkPAN testing is not. Every module uploaded to the CPAN and added to the index gets tested and analyzed and cross-referenced and, look at that, there's another piece of data added to the giant feedback loop which is community-driven software development. Every piece of DarkPAN code added stays in the dark, where none of this works...
... unless someone deliberately extracts relevant code from the DarkPAN and puts on the CPAN (which happens), or tracks bleadperl snapshots and testing releases and reports changes (which rarely happens), or files bugs after a release (which happens).
That's how community-driven software development works. If you're willing to work with the community in the open, we all benefit many times over.
There's one small problem, however. No one wants to break code in the DarkPAN. Note please that "break code" in the Perl 5 world is often a euphemism for establishing and following a deprecation policy. No one's advocating removing major features such as
AUTOLOAD, as tempting as that might be. Yet the specter of the DarkPAN often arises in discussions where it's the final nuclear deterrant against adding a simple piece of syntax to simplify a task which is more laborious it ought to be in a language under development in 2009. "We don't know what it would break," they say, and that ends the argument.
When I write software which matters, I create a test suite. I choose dependencies carefully. I never upgrade it in production without running the tests. (Sometimes I run the test suite just for the fun of it.) The function of that code matters, so I'm cautious.
If I upgrade a dependency, I run the test suite. If I find a bug, I add it to the test suite. If I upgrade my version of Perl, I run the test suite. In short, I take responsibility for identifying and diagnosing all potentially functional changes to the behavior of my code even if they come from external resources. It's my responsibility as maintainer of that code to keep it running, and I know of no better way than to maintain a comprehensive, sane test suite to verify that it maintains the behavior I intended.
I expect the people to whom I distribute the software to do the same, so I include the test suite there. It's common sense software management. It's common sense operations.
Not so with the DarkPAN. Somehow, the Perl world has decided that this unmeasurable, unknowable conglomeration of code which may or may not exist is incapable of performing software and operations management to a minimal degree of competence, despite the fact that, for example, this is what operating system vendors do. This is what professional system administrators do.
Thus the burden of integration and deployment falls not to the people who know where and what DarkPAN software exists and how it works and what it uses, but to the least appropriate group of people: people who cannot possibly see it and can only guess at what it does, what it needs, how it does it, and if any of that is appropriate or supported.
I can imagine that if I called technical support for a product, I wouldn't get support if I refused to disclose any information about my problem -- or even if I actually had a problem.
Why should anyone expect differently from volunteers?