war story: parallel(1) command

[Eric B]

> | From: Mauro Souza <thoriumbr-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>
>
> | I would say you should use crc32 instead of md5sum. But before saying
> that,
> | I made a simple test, hashing a 64MB video file:
...
> <https://groups.google.com/d/msg/net.sources/4ERvPT6oxdA/nY6T761u2h0J>
> That code made it into a couple of IETF RFCs.  I originally wrote it
> to check transfers from my Altair -- I had a version that was part of
> the ROM I wrote.  But there are some unfortunately easy collisions.
> For example, initial zero bytes make no difference to the result.

It is easy to find collisions on a Linux filesystem with a 32-bit CRC
checksum.  If you have more than 65,000 (~ 2^(32/2)) files,
you will probably find at least one.

One would think that MD5 is good enough,
but because it is cryptographically broken, you could find collisions
that were legitimately generated and not adversarial.
For example, you might unpack something related to hashes, and it
contains examples of two different files with duplicate MD5 hashes.

To be safe, use a stronger hash.


--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists