Compressed partition or block device?

[Jamon Camisso]

On 12-07-09 03:00 PM, Lennart Sorensen wrote:
> Deduplication on the other hand is huge.
> 
> Compression at the filesystem level rather than block device works better
> since the filesystem can take care of allocations for the compressed data,
> and the meta data can stay uncompressed.  There are at least patches
> around for ext2 that does this that are being maintained for current
> kernel versions.  e2compr for example.
> 
> But other than some 'fuse' program I don't know of anything that works
> at the block level since it is quite simply the wrong place to do it.
> You need to handle running out of space if the data doesn't compress
> which is much easier to do if the filesystem is aware of the compression
> (and in fact responsible for it).

There is a ZFS port that runs in kernel space, which is actively
maintained by Brian Behlendorf (of Apache, Mozilla fame). If someone was
going to port ZFS to Linux, he's one of the few I'd expect to be able to
do it correctly.

It works well since it is a port of ZFS proper. Consider this /sites/git
directory that I use on an EXT4 filesystem, it is 479MB:

git at zeus:pts/10:/git % du -sh /sites/git
479M    /sites/pwp

Now look at the same files (copied over with rsync) on a ZFS volume:

git at zeus:pts/10:/BACKUPS/git % zfs get compressratio BACKUPS/git
NAME         PROPERTY       VALUE  SOURCE
BACKUPS/git  compressratio  1.24x  -

git at zeus:pts/10:/BACKUPS/git % du -sh /BACKUPS/git
352M    /BACKUPS/git

That's a 127mb savings.

You can even use disk image files as your zpool devices, so you don't
need to use an entire disk or partition if you just want to try things out.

http://zfsonlinux.org/

Jamon
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists