Compressed partition or block device?

[Lennart Sorensen]

On Mon, Jul 09, 2012 at 06:26:55AM -0700, William Park wrote:
> I want to try "compressed partition or block device" (just
> like RAID or encryption layer) on top of which would go your
> regular filesystem.  Is there such thing?  Any keywords
> to get me started?

That probably exists but could be rather tricky.

After all you don't want to just compress the disk from one end to the
other, since then any change in the middle would require recompressing
everything after it.  So that means it needs more of a per block level
of compression, but now the question is how to place compressed blocks
on the disk in an efficient manner given they will vary in size.

Compressed read only is much simpler.  Just compress it and store it
and decompress pieces as needed on read.  Fragmentation and such is a
non issue.  Read only is of course not likely to be what you want.

To some extent due to the size of disks and the overhead and complexity
of compression, I really don't think anyone cares much for compression
at the block device anymore.

Deduplication on the other hand is huge.

Compression at the filesystem level rather than block device works better
since the filesystem can take care of allocations for the compressed data,
and the meta data can stay uncompressed.  There are at least patches
around for ext2 that does this that are being maintained for current
kernel versions.  e2compr for example.

But other than some 'fuse' program I don't know of anything that works
at the block level since it is quite simply the wrong place to do it.
You need to handle running out of space if the data doesn't compress
which is much easier to do if the filesystem is aware of the compression
(and in fact responsible for it).

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists