[GTALUG] File larger than partition? (Docker-related)

[Giles Orr]

I ran a backup today and noticed one file because it took so long to back
up.  I'm using FC25 on this machine, and used the OS packages of Docker.  I
seem to have a 100G file on a 12G partition:

root at toshi7:/var/lib/docker/devicemapper/devicemapper# ls -lh
total 34M
-rw------- 1 root root 100G Oct 25 22:02 data
-rw------- 1 root root 2.0G Oct 25 22:02 metadata
root at toshi7:/var/lib/docker/devicemapper/devicemapper# df -h
Filesystem              Size  Used Avail Use% Mounted on
...
/dev/sda8                16G   12G  3.8G  75% /
tmpfs                   7.8G  776M  7.1G  10% /tmp
/dev/mapper/home_crypt  237G  211G   24G  91% /home
...

/var/ is part of / , not a separate partition.  The file appears to be a
plain file, not a link or device.  How exactly did my backup process (rsync
to an external hard drive) manage to back up 100G of data from a 12G
partition?

"docker info" contains some very interesting output: I've included that
below.  The most obvious/naive solution would simply be to exclude
/var/lib/docker/devicemapper/devicemapper/ from my backups.  Because I'm
guessing I don't need a backup of the "Data loop file?"  Is that correct?
To my surprise, the file and its backup share an md5sum value - but then, I
haven't touched docker in the intervening time.  But I'd still really like
to know either where it's storing 100G of data, or what's going on if it's
not actually storing that much.  It's very weird, and makes it awfully hard
to calculate how big your backup will be when your partitions are evidently
behaving like Tardises.

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 8
Server Version: 1.12.6
Storage Driver: devicemapper
 Pool Name: docker-8:8-802519-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 27.79 MB
 Data Space Total: 107.4 GB
 Data Space Available: 4.943 GB
 Metadata Space Used: 618.5 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
 Thin Pool Minimum Free Space: 10.74 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production
use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage
device.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.136 (2016-11-05)
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: null host bridge overlay
Swarm: inactive
Runtimes: oci runc
Default Runtime: oci
Security Options: seccomp
Kernel Version: 4.12.11-200.fc25.x86_64
Operating System: Fedora 25 (Twenty Five)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 8
Total Memory: 15.56 GiB
Name: toshi7
ID: TJAV:PKWI:UYK2:UGAQ:GHUP:2HDO:4RBD:U54O:X7DM:SUPY:WQAM:IFTX
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8
Registries: docker.io (secure)

-- 
Giles
https://www.gilesorr.com/
gilesorr at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gtalug.org/pipermail/talk/attachments/20171120/78819a6d/attachment.html>