A Generation Lost in the Bazaar - Poul-Henning Kamp article

Tue Aug 21 20:06:05 UTC 2012

On Tue, Aug 21, 2012 at 10:33 AM, Jamon Camisso
<jamon.camisso-H217xnMUJC0sA/PxXw9srA at public.gmane.org> wrote:
> I have tremendous respect and admiration for Poul-Henning Kamp, and his
> most recent article in the ACM queue has only bolstered it.
>
> He discusses some of the failings of the Bazaar software development
> model championed by the likes of Eric Raymond. Given his background and
> his lifelong involvement with free software like FreeBSD and Varnish, I
> think much of what he has to say rings true.
>
> For example, he shows an autoconf macro and then explains: "That is the
> sorry reality of the bazaar Raymond praised in his book: a pile of old
> festering hacks, endlessly copied and pasted by a clueless generation of
> IT "professionals" who wouldn't recognize sound IT architecture if you
> hit them over the head with it."
>
> http://queue.acm.org/detail.cfm?id=2349257
>
> Worth a read.

And it's a bit unfair, too.  Getting code to a state where it *can* be
reused is a mighty difficult task.

We got Unix as a followup from a number of systems that might be
regarded as "failed" (which mightn't be entirely fair, but let me "go
with this"...)...

- MULTICS tried abstracting a lot of layers, and introduced the notion
of building an OS in a high level language (PL/1 is readily arguable
to be higher level than C!).

- C followed BCPL and B.

- Unix was originally written in assembly language, and redoing in C
was a later stage.

A lot of the things that are *good* about Unix are things that
required experimentation and, effectively, *failed* earlier attempts
that had to be discarded when implementing Unix.

I agree that we're getting some hugely messy systems these days.  I
have been working on some RPM "spec" files, and building something
that "works" seems to be a trial-and-error task where everyone I talk
with uses recipes that they have discovered they could use, as opposed
to knowing are intentionally valid.  The docs haven't guided me nearly
as much as I would have liked.  Is the result well-engineered?  Based
on the things I don't know, I'm pretty sure not.

I disagree with Kamp's assertion that a lack of understanding of
portability is the root of the problems; that's an expression of the
deeper problem that people haven't done quite enough abstraction to
understand what problems they were trying to solve, and how to
implement the solutions WELL.

That's a difficult set of tasks, and while the IT industry may well
have lots of idiots, the supposedly smarter, more ambitious folks
don't necessarily do better.  Ongoing standards efforts for C, C++,
and C Standard Library show off a whopping lot of messiness, too, and
they don't let idiots near those nearly as easily.  I watch the
goings-on surrounding Go with interest, as the implementers had
involvement with Unix and C, and seem to be trying to create something
cleaner.  (Interesting recent essay on this:
<http://blog.golang.org/2012/08/organizing-go-code.html>  I don't
think it explains the matter enough, but it's trying.)

I'm watching what's going on with Puppet with interest, but a bit of a
baleful eye, too.  One of the "doctrines" seems to be that everything
should be a defined package (which is why I'm writing RPM .spec files
;-)).  I get the feeling that the doctrine has, behind it, the
imagined principle that 'stuff that's managed via packages and thus
fit into a distribution is better/more maintainable/usable/...'

But the spawning (and, later, usually, deaths) of zillions of Linux
distributions seem to put some lie to this; if RPM .spec files become
a hot, in-language, I suspect a result of this is a lot of bad RPM
packages.

I see huge messes in two places, packaging (which Kamp observes on),
and libraries (which I think he only touches very indirectly).  Part
of the proliferation of zillions of packages, and of the proliferation
of dependencies (which Kamp notices, in his observation of the need
for graphics/libtiff as a dependency for firefox) results from
libraries being not thought through sufficiently carefully.

I presume that the graphics/libtiff dependency he observes results
from there being some other library that Firefox uses being one where
it proved attractive to include graphics/libtiff, because, while
Firefox mayn't really want to use .TIFF files, perhaps some other app
might, and might want that usage integrated into library
functionality.  I think we've "blown" something here, and it isn't
remotely just Firefox or libtiff that are affected.

If you want to build Postgres, at configuration time, you need to pick
from the following set of possible add-ons:

  --with-tcl              build Tcl modules (PL/Tcl)
  --with-perl             build Perl modules (PL/Perl)
  --with-python           build Python modules (PL/Python)
  --with-gssapi           build with GSSAPI support
  --with-krb5             build with Kerberos 5 support
  --with-pam              build with PAM support
  --with-ldap             build with LDAP support
  --with-bonjour          build with Bonjour support
  --with-openssl          build with OpenSSL support
  --with-selinux          build with SELinux support
  --without-readline      do not use GNU Readline nor BSD Libedit for editing
  --with-libedit-preferred
                          prefer BSD Libedit over GNU Readline
  --with-ossp-uuid        build contrib/uuid-ossp, requires OSSP UUID library
  --with-libxml           build with XML support
  --with-libxslt          use XSLT support when building contrib/xml2
  --without-zlib          do not use Zlib

None of these optional additions are evil, but, given this set of
options, there are 2^14*3, or 49152 different ways to configure
Postgres, and whatever way you choose, you have to pick at compile
time.

What is a distribution maker to do?  There are likely a *few* options
that can be implied from the system (e.g. - no value to inclusion of
SELinux support if running on FreeBSD), but just about any option left
off is liable to make enemies, and hence, if you install something
that depends on PostgreSQL, you're liable to be drawing in a series of
libraries that you mightn't have expected.

I think we've "blown it" in the sense that support for these sorts of
things are NOT sufficiently abstracted that you can build without
needing to pull in rather more dependencies than you wanted.  And
that's true for libraries and packages, both.

We may be suffering a language problem, as our typical implementation
languages, C and C++, have to statically resolve interfaces for the
above sorts of things at compile time.  The notable example of attempt
to define dynamic interfaces came in CORBA, with Dynamic / "Any"
interfaces.  See
<http://www.drdobbs.com/object-interconnections-dynamic-corba-pa/184403840>.
 The implementation of this in C is just heinous, and that may be the
root of some of the problem.

Android has an interesting notion for interfacing to extra things in
the form of what they call "Intents"
<http://android-developers.blogspot.ca/2012/02/share-with-intents.html>

I'm not sure I have more of a conclusion.

I don't think using Android is an answer to the "problems of grubby bazaars."

Go would be more nearly the shape of an answer, but reimplementing
everything in a new language is a crummy thing to need to do.

But *another* disease disease we have is the one of building things
into gigantic monolithic applications.  That's certainly a reasonable
way to characterize the "desktop" pieces like web browsers, most email
clients, the "integrated word processor/spreadsheet/presentation
package", and, for that matter, a lot of the Desktop Environment apps
for GNOME and KDE.  To build things that are more keeping with Unix
Philosophy might essentially require redesigning them, at which point,
reimplementing in a new language might not be the biggest part of the
task.
-- 
"If Ada became the hot, in-language  you would see a lot more bad code
in   Ada."
-- Thaddeus L.  Olczyk <olczyk-IKAnbrrkwKZdz2imjWt+ww at public.gmane.org>, comp.lang.C++
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists