[GTALUG] War Story: Asus UX305ca SSD failures

[D. Hugh Redelmeier]

| From: Alvin Starr via talk <talk at gtalug.org>

| It would be interesting to see the bit densities over time from stone
| tablets(how would cave paintings count?)  to the latest production storage
| systems.
| It would also be interesting to know how many people had access to the storage
| media over time. That would have started with a few priests and specially
| trained people to just about everybody having a cell phone with a few GB of
| storage.

Yeah.  The main reason for new formats is that they are cheaper or
larger. Sadly not because of endurance.

One McLuhan-esqe observation is that what gets recorded on new media
is likely less significant / important.  My brother often remarks that
that photos on film (or glass and metal plates!) were way more
considered and significant than photos now.  We are buried in snaps
and will need AI to find ones that we might actually be interested in
later.

A separate problem is that we often don't know how useful or
interesting something is until later.  I imagine that to collectors
TV Guide is more precious than National Geographic since nobody saved
the former and everybody saved the latter.

(It is thought that the NSA is buried in data with insufficient means
to find even the obviously interesting stuff.)


| From: James Knott via talk <talk at gtalug.org>

| On 2019-08-09 01:19 PM, Russell Reiter wrote:
| > Jpegs are an exported file format created from aggregated image data
| > collected by the CCD. They are digital files and subject to
| > transmission errors just like any other signal. My 13 megapixel phone
| > saves image data directly as a jpg file. Sure the raw data has been
| > manipulated before writing the original, but that is much different
| > than having an image recorded using raw format and then exporting a
| > copy in a lossy format in order to save space. 

In cameras, it seems to be called RAW, not raw.  I think that is to
remind us that each camera has its own proprietary format.  Only
sometimes is the format disclosed.

Worse: some cameras never have a raw image: they convert to JPEG on
the fly since their storage has neither the bandwidth or capacity to
store a whole raw image.

| Perhaps you need to learn about some of the technology.  Some media,
| such as CDs & DVDs use forward error correction, to ensure data is
| copied correctly.

A significant number of problems cannot be corrected by FEC.  In
particular, misplacing a CD or having a 100% failure in an SSD (the
original problem that prompted the thread).

|   When you transfer data over a network, there is a
| checksum used with TCP, with IP(v4) and a CRC check at the Ethernet
| level.  On top of that some applications provide their own integrity
| check.  So, it would be very difficult for an error to propagate.  Now,

An excuse to roll out another story.

When I was trying to make my Altair useful, I wrote a monitor.  I
needed a way of doing integrity checks for files recorded to audio
tape and files sent over a serial line.  I decided that CRC16 was way
better than a simple XOR checksum.  I looked into it and devised a way
of calculating CRCs in software by byte-at-a-time operations rather
than bit at a time (CRC was designed to be implemented in hardware,
bit-wise, with a linear-feedback shift register).

I wrote the code in 8080 assembler and in C.

I released the code on usenet (no, not all of usenet was porn).

Unbknownst to me, my code made it into the RFCs for PPP.  Open source
works.  <https://tools.ietf.org/html/rfc1134>

BTW, CRC isn't any good at detecting forgeries.  For that we have
cryptographic hashes.  And those hashes need to be protected against
forgery too (digital signatures or another secure communications channel).