Reliability of USB-based filesystems? Lack thereof?

[Lennart Sorensen]

On Wed, Nov 05, 2008 at 01:49:54PM -0500, Christopher Browne wrote:
> I have been running into a problem lately which I thought I'd "toss
> over the wall" to see if others have experienced similar...
> 
> I have been having *terrible* troubles trying to copy files of any
> material size (e.g. - such as videos and such) onto USB-based devices,
> notably USB memory sticks and onto SD cards mounted via USB.
> 
> I copy files over, and there are two tendancies:
> 
> 1.  Frequently, I run out of disk space on the device, even though
> there's certainly plenty of room left.
> 
> 2.  After completing copying, I unmount the filesystem (which takes a
> while, due to flushing out updates), and when I remount, the
> filesystem tends to be pretty corrupt, with wacky directory entries
> containing control characters.
> 
> I'm not at all sure that this is a USB-relevant issue; what I am
> getting mighty suspicious of is that perhaps I'm blowing up the VFAT
> filesystem (fyi, I keep having to rebuild the filesystems when I
> attempt a retry).
> 
> All seems to be OK if I keep to moderate file sizes (e.g. - a few MB
> in size); everything seems to go bad when file size grows to 100's of
> megabytes.  That's not formal "troubleshooting info;" that's more like
> an anecdote.
> 
> What seems rather interesting is that I observe this in a
> cross-platform fashion; I see this phenomenon both on Linux and on
> MacOS.
> 
> A quick google finds the following (reported on Darwin, I'm not sure
> if this is with "Hackintosh" or with "just Darwin"; don't think that's
> too relevant...)
> http://forum.insanelymac.com/index.php?s=fd9e8e56f8887e542acf2e01e7ea6917&showtopic=57523&st=0&p=410781&#entry410781
> 
> It is suggested that the problem might be insufficient power to the
> USB devices; that seems implausible when I'm mounting them on a
> desktop box where there should surely be plenty of power.
> 
> I have been seeing this phenomenon for a while, on a variety of hardware.

Remember that FAT16 filesystems (used on cards 2GB and under normally)
has a limited number of directory entries in the root dir.  Long file
names (as VFAT does) uses multiple entries per file.  This means you can
run out if directory entries very quickly, if you place them all in the
root.  If you create a subdir, you can put all that you want in that
subdir, since it isn't limited in size and hence the number of entries.

So that's a filesystem design limitation.

FAT32 does not have this limitation, but SD card spec only says to use
FAT32 for SDHC cards (which is normally 4GB+ cards).

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists