poor Raid5 write speed

[Lennart Sorensen]

On Tue, Jul 27, 2010 at 05:09:04PM -0400, William Park wrote:
> chunksize = 64kB

Sounds decent.

> > How many disks in the raid?
> 
> 4 x 1TB

Well I run 4 1TB WD caviar black drives, so that seems similar.

> mkfs.ext4 -m0.1 -E stride=16,stripe-width=48
> where
>     stride = chunksize / blocksize = 64kB/4kB = 16
>     stripe-width = stride * (4 - 1) = 16 * 3 = 48

I use ext3 still, but I think I used very similar settings.

> nVidia MP55 (nForce 570 Ultra, Asus M2N-E).

Mine is an ICH9 controller.

> Single-core Athlon64 LE-1620 (2.4GHz, 1MB, 45W).
> DDR2-667 (AM2 special, 8GB)

I have a Core 2 Quad Q6600.  So a significantly higher amount of CPU
power to run the raid.

> # hdparm -tT /dev/sd[bcdef] /dev/md5
> 
> /dev/sdb:
>  Timing cached reads:   1684 MB in  2.00 seconds = 842.19 MB/sec
>  Timing buffered disk reads:  392 MB in  3.00 seconds = 130.63 MB/sec
> 
> /dev/sdc:
>  Timing cached reads:   1756 MB in  2.00 seconds = 877.90 MB/sec
>  Timing buffered disk reads:  342 MB in  3.01 seconds = 113.56 MB/sec
> 
> /dev/sdd:
>  Timing cached reads:   1738 MB in  2.00 seconds = 868.45 MB/sec
>  Timing buffered disk reads:  288 MB in  3.00 seconds =  95.98 MB/sec
> 
> /dev/sde:
>  Timing cached reads:   1842 MB in  2.00 seconds = 919.92 MB/sec
>  Timing buffered disk reads:  300 MB in  3.02 seconds =  99.23 MB/sec
> 
> /dev/sdf:
>  Timing cached reads:   1790 MB in  2.00 seconds = 895.27 MB/sec
>  Timing buffered disk reads:  358 MB in  3.00 seconds = 119.19 MB/sec
> 
> /dev/md5:
>  Timing cached reads:   1706 MB in  2.00 seconds = 852.76 MB/sec
>  Timing buffered disk reads:  786 MB in  3.00 seconds = 261.62 MB/sec
> 
> /dev/sdb is my root disk, and it does 130MB/s for both sequential read
> and write.  That's good.  /dev/md5 is Raid5 over /dev/sd[cdef].  Its
> sequential read is 260MB/s, but sequential write is 110MB/s.  Now,
> that's bad.

Well I haven't tested writing, but 260MB/s read is perfectly fine.
For writing I get around 100MB/s according to dd.  I am not sure what
the raw write speed of the disks are.  It is certainly lower than the
read speed.  ext3 isn't exactly known for great write performance
after all, but neither are harddisks in general compared to reading.

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists