Q: Mailbox format

[Henry Spencer]

On Tue, 27 Apr 2004, Taavi Burns wrote:
> > Not necessarily.  Improved optimization for lots of small files is something
> > that could reasonably happen to the code supporting most any filesystem.
> 
> It could...but then you'd either have to go through huge data transformations
> in memory and ensure that it syncs with disk all the time...

Not really.  With careful thought, upward-compatible upgrades are often
possible.  (Not always, but often.)  Data format often can stay exactly the 
same during improvements.

For example, maintaining on-disk consistency in a traditional filesystem
design does *NOT*, repeat *NOT*, require that changes be written out
immediately -- it only requires that changes be written out in the right
order.  More precisely, it requires that the order in which changes are
written out must observe certain constraints.  Tracking the order
constraints is more complicated than writing things out immediately, but
it's not prohibitive, and performance is vastly better with no change in
on-disk data format whatsoever.  "Work smarter, not harder."

Even when on-disk format changes are needed, cleverness often permits
doing them in an upward-compatible way.

> Traditional UNIX filsystems just don't handle thousands of files very well,
> since they store directories as flat files.  Really, that's silly...

Rather, it's a tradeoff that doesn't scale up well.  But fixing it can
usually be done without tearing everything up and starting over.  Just
takes some effort and some intelligence. 

Forcing people to abandon their old filesystems, tools, etc. and start
over just to get performance gains is stupid, inconsiderate, and usually
unnecessary. 

                                                          Henry Spencer
                                                       henry-lqW1N6Cllo0sV2N9l4h3zg at public.gmane.org

--
The Toronto Linux Users Group.      Meetings: http://tlug.ss.org
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://tlug.ss.org/subscribe.shtml