[GTALUG] On the subject of backups.

[Lennart Sorensen]

On Mon, May 04, 2020 at 04:38:28PM +0200, ac via talk wrote:
> Hi Alvin,
> 
> On a 2TB dataset, with +-600k files, I have piped tree to less with 
> limited joy, it took a few hours and at least I could search for
> what I was looking for... - 15TB and 100M is another animal though
> and as disk i/o will be your bottleneck, anything will take long, no?
> 
> now, for my own info/interest, can you tell me which fs is used for this
> ext3? 

Hmm, sounds awful slow.

Just for fun I ran find on one of my drives:

# time find /data | wc -l
1825463
real    3m57s.208s

That is with 5.3T used out of 6.0TB.

Running it a second time when it is cached takes 7.7s.  Tree takes 14.7s.

Another volume:
# time find /mythdata | wc -l
54972

real    0m1.924s

That is with 15 TB out of 15 TB in use (yes that one always fills up
for some reason).

Both of those are lvm volumes with ext4 on top of software raid6 using
5400rpm WD red drives.

Seems either XFS is unbelievable bad, or there isn't enough ram to cache
the filesystem metadata if you are having a problem with 100M files.
I only have a measly 32GB in my home machine.

-- 
Len Sorensen