A Generation Lost in the Bazaar - Poul-Henning Kamp article

Wed Aug 22 14:56:13 UTC 2012

On Tue, Aug 21, 2012 at 04:06:05PM -0400, Christopher Browne wrote:
> And it's a bit unfair, too.  Getting code to a state where it *can* be
> reused is a mighty difficult task.
> 
> We got Unix as a followup from a number of systems that might be
> regarded as "failed" (which mightn't be entirely fair, but let me "go
> with this"...)...
> 
> - MULTICS tried abstracting a lot of layers, and introduced the notion
> of building an OS in a high level language (PL/1 is readily arguable
> to be higher level than C!).
> 
> - C followed BCPL and B.
> 
> - Unix was originally written in assembly language, and redoing in C
> was a later stage.

Well eventually rewriting assembly again and again gets to be a bit
annoying.

> A lot of the things that are *good* about Unix are things that
> required experimentation and, effectively, *failed* earlier attempts
> that had to be discarded when implementing Unix.

Well the system call list sure hasn't changed much since the begining
so at least the main API was rather well thought out from the start.

> I agree that we're getting some hugely messy systems these days.  I
> have been working on some RPM "spec" files, and building something
> that "works" seems to be a trial-and-error task where everyone I talk
> with uses recipes that they have discovered they could use, as opposed
> to knowing are intentionally valid.  The docs haven't guided me nearly
> as much as I would have liked.  Is the result well-engineered?  Based
> on the things I don't know, I'm pretty sure not.

I think .spec files for RPM is a design mistake.  I much prefer dealing
with debian packages.  RPM was a vast improvement over slackware's crappy
packaging system, but it is far from the well thought out design that
.deb is.

> I disagree with Kamp's assertion that a lack of understanding of
> portability is the root of the problems; that's an expression of the
> deeper problem that people haven't done quite enough abstraction to
> understand what problems they were trying to solve, and how to
> implement the solutions WELL.

If you insist on having things portable then you essentially have to
insist that APIs on systems can never change, so new development has
to stop.  That doesn't sound desirable.  Change for the sake of change
(which surprisingly Windows seems very fond of, while still trying to
maintain backwards compatibility) is certainly not a good idea.

> That's a difficult set of tasks, and while the IT industry may well
> have lots of idiots, the supposedly smarter, more ambitious folks
> don't necessarily do better.  Ongoing standards efforts for C, C++,
> and C Standard Library show off a whopping lot of messiness, too, and
> they don't let idiots near those nearly as easily.  I watch the
> goings-on surrounding Go with interest, as the implementers had
> involvement with Unix and C, and seem to be trying to create something
> cleaner.  (Interesting recent essay on this:
> <http://blog.golang.org/2012/08/organizing-go-code.html>  I don't
> think it explains the matter enough, but it's trying.)

C++ and the STL are a huge mess.  I think both should be avoided.  C does
what it was meant to do, and should be handled with the care it requires
as a result.

> I'm watching what's going on with Puppet with interest, but a bit of a
> baleful eye, too.  One of the "doctrines" seems to be that everything
> should be a defined package (which is why I'm writing RPM .spec files
> ;-)).  I get the feeling that the doctrine has, behind it, the
> imagined principle that 'stuff that's managed via packages and thus
> fit into a distribution is better/more maintainable/usable/...'
> 
> But the spawning (and, later, usually, deaths) of zillions of Linux
> distributions seem to put some lie to this; if RPM .spec files become
> a hot, in-language, I suspect a result of this is a lot of bad RPM
> packages.

Well if you start out choosing to use RPM, then you will certainly end
up with a lot of messy packages.  Those are way too hard to make well.

> I see huge messes in two places, packaging (which Kamp observes on),
> and libraries (which I think he only touches very indirectly).  Part
> of the proliferation of zillions of packages, and of the proliferation
> of dependencies (which Kamp notices, in his observation of the need
> for graphics/libtiff as a dependency for firefox) results from
> libraries being not thought through sufficiently carefully.
> 
> I presume that the graphics/libtiff dependency he observes results
> from there being some other library that Firefox uses being one where
> it proved attractive to include graphics/libtiff, because, while
> Firefox mayn't really want to use .TIFF files, perhaps some other app
> might, and might want that usage integrated into library
> functionality.  I think we've "blown" something here, and it isn't
> remotely just Firefox or libtiff that are affected.

I think it is an indication of the library having been done right.  Make a
library that handles graphics formats and use the standard libraries
for each format.  Who cares if firefox doesn't want one of the formats,
some other program does want it and they can use the same library.
That's a good thing.

> If you want to build Postgres, at configuration time, you need to pick
> from the following set of possible add-ons:
> 
>   --with-tcl              build Tcl modules (PL/Tcl)
>   --with-perl             build Perl modules (PL/Perl)
>   --with-python           build Python modules (PL/Python)
>   --with-gssapi           build with GSSAPI support
>   --with-krb5             build with Kerberos 5 support
>   --with-pam              build with PAM support
>   --with-ldap             build with LDAP support
>   --with-bonjour          build with Bonjour support
>   --with-openssl          build with OpenSSL support
>   --with-selinux          build with SELinux support
>   --without-readline      do not use GNU Readline nor BSD Libedit for editing
>   --with-libedit-preferred
>                           prefer BSD Libedit over GNU Readline
>   --with-ossp-uuid        build contrib/uuid-ossp, requires OSSP UUID library
>   --with-libxml           build with XML support
>   --with-libxslt          use XSLT support when building contrib/xml2
>   --without-zlib          do not use Zlib
> 
> None of these optional additions are evil, but, given this set of
> options, there are 2^14*3, or 49152 different ways to configure
> Postgres, and whatever way you choose, you have to pick at compile
> time.
> 
> What is a distribution maker to do?  There are likely a *few* options
> that can be implied from the system (e.g. - no value to inclusion of
> SELinux support if running on FreeBSD), but just about any option left
> off is liable to make enemies, and hence, if you install something
> that depends on PostgreSQL, you're liable to be drawing in a series of
> libraries that you mightn't have expected.

Why not just turn on everything and don't worry about it?

Or in the case of php, debian modularizes it and has packages for lots
of the extensions for php as seperate packages that you can install if
you want them and not install if you don't.  Plugins can work, but does
require extra coding effort.

> I think we've "blown it" in the sense that support for these sorts of
> things are NOT sufficiently abstracted that you can build without
> needing to pull in rather more dependencies than you wanted.  And
> that's true for libraries and packages, both.

Why are you building things yourself in the first place?

> We may be suffering a language problem, as our typical implementation
> languages, C and C++, have to statically resolve interfaces for the
> above sorts of things at compile time.  The notable example of attempt
> to define dynamic interfaces came in CORBA, with Dynamic / "Any"
> interfaces.  See
> <http://www.drdobbs.com/object-interconnections-dynamic-corba-pa/184403840>.
>  The implementation of this in C is just heinous, and that may be the
> root of some of the problem.
> 
> Android has an interesting notion for interfacing to extra things in
> the form of what they call "Intents"
> <http://android-developers.blogspot.ca/2012/02/share-with-intents.html>

It also uses java.

> I'm not sure I have more of a conclusion.
> 
> I don't think using Android is an answer to the "problems of grubby bazaars."
> 
> Go would be more nearly the shape of an answer, but reimplementing
> everything in a new language is a crummy thing to need to do.

New languages don't solve bad design by bad developers.

> But *another* disease disease we have is the one of building things
> into gigantic monolithic applications.  That's certainly a reasonable
> way to characterize the "desktop" pieces like web browsers, most email
> clients, the "integrated word processor/spreadsheet/presentation
> package", and, for that matter, a lot of the Desktop Environment apps
> for GNOME and KDE.  To build things that are more keeping with Unix
> Philosophy might essentially require redesigning them, at which point,
> reimplementing in a new language might not be the biggest part of the
> task.

Well certainly kde consists of thousands of programs.  It is not
monolithic at all.  It even consists of dozens if not hundreds of seperate
source packages.

firefox is a huge monolithic beast (and chrome even worse).

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists