[GTALUG] File larger than partition? (Docker-related)

[Lennart Sorensen]

On Mon, Nov 20, 2017 at 05:25:16PM +0000, Giles Orr via talk wrote:
> I ran a backup today and noticed one file because it took so long to back
> up.  I'm using FC25 on this machine, and used the OS packages of Docker.  I
> seem to have a 100G file on a 12G partition:
> 
> root at toshi7:/var/lib/docker/devicemapper/devicemapper# ls -lh
> total 34M
> -rw------- 1 root root 100G Oct 25 22:02 data
> -rw------- 1 root root 2.0G Oct 25 22:02 metadata
> root at toshi7:/var/lib/docker/devicemapper/devicemapper# df -h
> Filesystem              Size  Used Avail Use% Mounted on
> ...
> /dev/sda8                16G   12G  3.8G  75% /
> tmpfs                   7.8G  776M  7.1G  10% /tmp
> /dev/mapper/home_crypt  237G  211G   24G  91% /home
> ...
> 
> /var/ is part of / , not a separate partition.  The file appears to be a
> plain file, not a link or device.  How exactly did my backup process (rsync
> to an external hard drive) manage to back up 100G of data from a 12G
> partition?

Probably a sparse file.  Telling rsync to handle sparse files efficiently
will help a lot.  If you don't tell it, it will expand the sparse file
with the unwritten parts being all zeros.

Often disk images for VMs are sparse files (so unwritten parts are not
actually allocated yet).

-- 
Len Sorensen