war story: some USB flash memories are very slow

[D. Hugh Redelmeier]

TL;DR:

When dding to a raw USB flash memory, you can save perhaps half the
time by:

1) using a reasonably sized buffer.  1M seems enough.
2) using direct I/O.

Example:
	dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi oflag=direct bs=16M && eject /dev/sdi


The Story
=========

The "normal" way of installing Fedora is via DVD.  There are other ways, 
such as over the LAN.

I don't want to waste DVDs, so I like to use a USB flash memory stick.

Today I downloaded and installed the Fedora 20 beta.  This was the install 
DVD image, not the live image.

I dded the image to a USB stick.  This seems to be the way you have to do 
it for this image.  The dd took 36 minutes for 4.6 GB.  That seems very 
slow to me.  That's about 20k / second.

The command was:
	time sudo dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi
The time might be an underestimate because some blocks were probably not 
yet flushed to the stick.

Should I have specified a (larger) block size?  I assume that caching 
makes that moot.

[enter Science Mode]

Apparatus:

- desktop computer:  I7-4770 processor, 24G RAM, Fedora 19 desktop with 
  some firefox windows/tabs open

- source file is on an SSD.  In fact everything is.

- USB stick: Kingston 8GB DataTraveler 101.  Not known for speed.

Should I have used the direct I/O and a larger block size?
This command took only 20 minutes:
	time sudo dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi oflag=direct bs=1M
Seems to make a significant difference.

Is caching distorting things?
Right after the previous command, I did the following:
	time sudo dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi bs=1M
and it took only 12 seconds!
But then I did:
	time sync
and that took 31.5 minutes!
So caching significantly distorts timing.


Time for fresh experiments.

su for these (to avoid possible sudo password delays).

Eject USB drive & reinsert, clearing all caching:
	eject /dev/sdi
	# remove and re-insert USB drive
	umount /dev/sdi1

Time the naive dd:
	time dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi ; time eject /dev/sdi
real    38m7.526s
real    0m0.254s
Note: almost no time required for flushing the cache (eject must do
that).  Quite surprising.

Re-insert drive & unmount.

Time the naive dd with a larger buffer:
	time dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi bs=1M ; time eject /dev/sdi
real    31m54.402s
real    0m0.733s
Note: larger buffer made a difference even when going through buffer
cache.

Re-insert drive & unmount.

time the direct I/O version with a larger buffer:
	time dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi oflag=direct bs=1M ; time eject /dev/sdi
real    20m7.544s
real    0m0.003s
Note: direct I/O made a significant difference.

Re-insert drive & unmount.

time the direct I/O version with a still larger buffer:
	time dd if=Fedora-20-Beta-x86_64-DVD.iso of=/dev/sdi oflag=direct bs=16M ; time eject /dev/sdi
real    20m7.903s
real    0m0.004s
Note: the still larger buffer made no difference with direct I/O.

I'm too lazy to try smaller sizes: I can easily spare 1M of RAM for
this command.
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists