Less XS/More Ctypes


Perl is in better shape now than ever, but there's much left to improve. (I wrote Five Features Perl 5 Needs Now almost two years ago, and four of five are available in whole or in sufficient substitute, so that's great progress.)

Rather than criticize Perl for what's going wrong (as if I were any position to do so), I prefer to suggest new areas of improvement.

One of the biggest possible improvements to the world of Perl 5 is a replacement for many cases of XS. Ryan Jdoubi's Ctypes for Perl was on my original list, and it's still on my list.

Why is this important?

XS can be inscrutable. Try reading the documentation, spread out between perlxstut and perlapi, with liberal use of the Cozens and Jenness book Extending and Embedding Perl and, ultimately, trawling through—even copying and pasting—chunks of the Perl 5 core which seems to do what you want.

Now debug typemap problems, problems with contexts in threaded versus unthreaded builds, and the vagaries of Makefile syntax across various platforms, not to mention the joy of detecting whether you have the right version of a shared library and its development headers on a version of an operating system you've never seen, let alone used, and (oh yes) Windows.

It's no wonder people want pure-Perl modules when possible.

It's also no wonder that, with all of the copying and pasting and reading of the Perl 5 source code in lieu of a working, understandable example of real-life XS code we can modify to our own purposes, far too many CPAN modules and an unkonwn and scary amount of DarkPAN modules abuse the Perl 5 internals in such a way that upgrading to a newer version of Perl seems like soul-swallowing black magic.

You try refactoring the internals sometime, then watch the black smoke come in from the next monthly bleadperl release and sob softly as you put back all of the mess and hope for the best.

If Perl 5's internals have much hope of refactoring and cleanup and sanity, the less XS code the better. Ctypes for Perl can help by reducing the need for XS. If most of the XS modules which are XS only because there was previously no other obviously better way to pass data between Perl and shared libraries, migrating them to Ctypes can remove the need for XS. Improving the interface between Perl and C is much easier because only Ctypes needs to change.

We won't entirely get rid of XS, but we can minimize its use to those projects which really, really need it The fewer of those, the easier they are to audit and review and port, if necessary.

The cleaner the internals of Perl 5 (and the better its extension interfaces in sanity, cleanliness, safety, composability, and documentation), the more possibilities for further development and improvements.

(I also expect that modules which use Ctypes instead of XS will be smaller, faster, easier to write, less memory hungry, and easier to install and to redistribute.)


Do you still think we need mod_perlite now that we have Plack?

Yes, in the sense that Plack deployment currently isn't as easy as PHP deployment.

No, in the hope that more and more very cheap web hosts will appear with CPAN- and Plack-friendly services now that Plack exists.

(I learned how to be non-committal from Larry himself.)

What is Ctypes For Perl, and where can one learn more about it? The link above simply says, "Call and wrap native C functions and libraries using only pure Perl!", which sounds intriguing. What's it do, what's it look like, how's it work?


Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

affiliated with ModernPerl.net



About this Entry

This page contains a single entry by chromatic published on September 3, 2010 1:55 PM.

What's Going Right in Perl was the previous entry in this blog.

Essential Philosophy for Perl 5 Novices is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Sponsored by Blender Recipe Reviews and the Trendshare how to invest guide

Powered by the Perl programming language

what is programming?