file checksum?

[David Thornton]

My post from Jul 28th to this same list:

I hashed a 2GB file 100 times with each of the digests available via
openssl, for a total of 1000 runs.

Average elapse time , seconds:

md5  Average =  0.7401 <------THIS is the fastest
mdc2  Average =  46.8681
rmd160  Average =  2.2449
sha  Average =  4.06665
sha1  Average =  1.3751 <- close second.
sha224  Average =  3.6005
sha256  Average =  3.6019 <-
sha384  Average =  6.7991
sha512  Average =  6.8885

I shuffled the runs hoping to "even out" caching.

crappy little low power Intel(R) Atom(TM) CPU  230   @ 1.60GHz

Also:

http://stackoverflow.com/questions/157998/whats-the-difference-between-sha-and-md5-in-php

As an aside I'd like to suggest looking into ZFS, a files system that
ensures that data is not corrupt on disk, and can transparently check and
recover from "bit rot".

Linux implementation of ZFS are not quite production ready but BSD and
Solaris offspring are. There are great "Storage applinaces" that allow you
to get at the greatness of ZFS without too much trouble:

That I know of and have used:

FreeNAS (BSD Based)
and
Nexenta (Open Indiana based)

David




On Mon, Sep 30, 2013 at 11:13 AM, William Park <opengeometry-FFYn/CNdgSA at public.gmane.org>wrote:

> Hi,
>
> I would like to do some kind of "checksum" on files (full or partial
> content) in order to catch unwanted changes (accidental or malicious).
> How would you do it?
>
> So far, I found
>     - MD/SHA digests from OpenSSL -- I'm worried about speed, and being
>       dependent on yet another library.
>     - crypt() from glibc -- It can do MD5/SHA, but it has to be a single
>       string.  It can't do multiple strings.
>
> Is there user-callable CRC routines in glibc?  Curiously, I can't find
> one, even though I'm told that TCP stack uses it internally.
> --
> William
> --
> The Toronto Linux Users Group.      Meetings: http://gtalug.org/
> TLUG requests: Linux topics, No HTML, wrap text below 80 columns
> How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gtalug.org/pipermail/legacy/attachments/20131002/193cb152/attachment.html>