Linux: so what's happening?

(written between 9/2/04 and 21/3/04, last updated 16/5/04)

Well, the 2.6 series kernel is here. It's arrived without too much fanfare, in the end, and though it offers quite a few improvements on the 2.4 series I suspect there's not much reason for average-joe-linux-user to upgrade.

Why is that?

Well, some of the main benefits of 2.6 manifest as new interfaces to user-space programs. For instance, epoll, kernel support for asynchronous i/o, and (my personal favorite) futexes and better kernel support for threading, which allows a much closer-to-posix and more efficient implementation of posix threads.

So what's the problem? Well, support for all these things requires support at the user level; in most cases that means support in glibc, the GNU C library. Actually, in the case of epoll (and perhaps aio, but read on) the support can easily be implemented in a seperate library. But in the case of the thread support, it really needs to be tied pretty closely to glibc.

Some history...

"linuxthreads" is the "current" version of the library implementing posix threads, or rather an approximation of posix threads, on linux. One of the main deviations from the posix standard when using linuxthreads is that each thread gets is own pid (process id). This also has implications for signal handling, seeing as signals are always directed to a pid. Thus it's not really possible to send a signal to a "thread group" as it should be (according to posix).

A while ago some folks at IBM came up with NGPT, or "Next Generation Posix Threads". See this link. NGPT used a M-on-N threading model with kernel modifications (against the 2.4 series kernel; the necessary changes are incorporated in 2.6 mainline) to provide a threading library with near-full posix compliance, useable as a drop-in replacement for pthreads in glibc-2.2 series (but not 2.3).

Around about the same time some folks at Redhat were working on something which they dubbed NPTL, the "New Posix Threading Library". See kerneltrap, lwn articles.

Redhat is also effectively the prime developer/maintainer of glibc, and it was probably partly due to this that NPTL and not NGPT effectively won out here. See the announcement from the NGPT team.

The present...

So now what do we have?

NGPT exists, and presumably will work with the 2.6 series kernel, but only with the 2.2 series of glibc. It appears to have been more-or-less abandoned by IBM.

The glibc-2.3 series has a few significant improvements over 2.2 so it seems reasonable to expect that NGPT's lack of support for 2.3 could be enough to be its undoing.

If you want to use glibc-2.3+, you have NPTL (or of course the original linuxthreads). Here's where it starts to get curious:

The latest official glibc is 2.3.2, and it doesn't include NPTL;
NPTL has not officially been released as a stable, seperate add-on;
Current versions of NPTL require CVS versions of glibc;
CVS versions of glibc + NPTL require __thread support in gcc;
gcc requires CVS versions of binutils in order to support __thread;
binutils, glibc, and nptl are hosted by Redhat;
Redhat was the first distributor to offer a distribution based on NPTL.

There's a few things I don't like about this. I want to be very careful and not say anthing too bad about Redhat, but when you look at the facts their control on the development of linux in general is quite strong.

What really starts to get scary is that glibc CVS at the moment has started to include references to glibc-2.3.4 (symbols are being versioned as GLIBC_2_3_4, and the Changelog file mentions the "2.3.3 release"). Now, as glibc-2.3.3 has not yet been released, how could this be? It's almost as if the maintainers have got their own glibc-2.3.3 release, but have kept it a secret from the rest of the world.

See this post on the "linux from scratch" archive. From this and several other mails in the same thread, it appears that some guys from LFS got wind of some news that Redhat may not be releasing tarballs of glibc in the future... I'm not sure exactly what this is supposed to mean... No glibc-2.3.3 ever?

When Redhat's doing all the work and dissing out the dough, it's hard to blame them if they don't feel like making a "stable" release. But I still have an itching feeling that something is wrong about all of this.

The saga continues

I recently stumbled across some further clues as to what is going, on the "libc-alpha" mailing list (hosted by Redhat):

2004-03 msg76
This is Ulrich Drepper's response to someone having a problem with the CVS version of glibc - essentially saying that he would rather not have "random people" trying to compile glibc in the first place. The entire thread of resulting posts is very interesting, in particular this one (again from Ulrich Drepper):
2004-03 msg78
... and the best yet, the one where Ulrich actually justifies his previous statements:
2004-03 msg114

Now, I will say that I think the points that Ulrich makes here are quite valid. It would seem, however, to be prudent to announce his and other maintainers intentions in terms of lack of new releases - if the rumours are true - and also to indicate the poor state of the build documentation. The perfect place to do this would be the glibc webpage.

What concerns me now, more than anything else, is not who has control of glibc, but rather the state of its documentation... Ulrich quite rightly points out that it's not his responsibility and no-one has the right to tell him or any other glibc maintainer what to do but I'm more than a little concerned about what this means for glibc:

Lack of documentation indicates lack of process. Lack of process generally leads to poor(er) quality software. These are well established ideas in the world of software engineering.

Never minding that it makes it damn difficult for outsiders to contribute to the software. This, I think, is what rankles the most; it's against the idea of "free" software - what use is undocumented code, that requires some particular (undocumented) version of binutils and compiler?

And the real question now is, what's to be done about it?

Note, 16/5/04. Glibc maintainers are completely ignoring any question on the libc-alpha mailing list regarding the next release of glibc. Ulrich Drepper has bluntly told other non-elites who ventured to post to the list to "take their questions elsewhere". While I sympathise with any maintainer of such a large and important piece of software, I am saddened and worried about glibc.