[GTALUG] long war story: growing the ESP (/boot/efi)

[D. Hugh Redelmeier]

| From: Alvin Starr via talk <talk at gtalug.org>

| Years ago I worked on an avionics project where we were storing data in an
| NVRAM.
| We had to write code to manage the wear-out in the cells and if I remember
| correctly the wear-out was in the hundreds of thousands to millions of writes
| before problems would occur.
| 
| I could not see normal usage causing wear-out unless someone were to have some
| kind of bug causing a high write rate.

The following is based on knowledge plus a fair bit of superstitious
extrapolation and interpolation.

NAND flash and NOR flash are different.  You are surely talking about
a NOR flash

Lennart is talking about an unknown system.  I suspect it used NAND
flash, if the description is right.

NOR flash:

- allows update of each byte separately (most are 1 byte wide)

- each update only wears the byte updated

- are small (eg. 8M is large).  Older consumer routers used NOR flash
  and that is why their flash sizes were so limited.

- it is reasonable to run programs directly out of NOR flash (it looks
  like RAM)

NAND flash

- allows updates in pages, but only to previously erased pages

- erases can only be done in large blocks.  The size of these blocks is
  generally secret but I think that they are in the range of 128KiB to
  1MiB.

- are so alien to deal with that they have a complicated controller
  between them and your system.  The controller emulates a disk drive.
  This does wear levelling.  There is some work at moving this into
  the OS for performance reasons, but it is not common and only for
  data-centres.

- density is everything in NAND flash systems.  The original ones used
  a single bit per cell (SLC) but now 3 bits per cell is common.  To store 3
  bits per cell, 8 charge levels must be distinguished.  This makes
  them much slower and less reliable (shorter-lived).

- each cell can only be erased a small number of times (perhaps 1000)
  but the wear-levelling usually prevents this being a big problem.
  It is not part of published specifications.

- consumer routers now use NAND flash for firmware and allow orders
  of magnitude larger firmware.  For reasons I haven't investigated,
  it took OpenWRT years to support NAND flash routers.

- it is unreasonable to run programs directly out of NAND flash.  They
  can, however, be paged in, just as one would do from a HDD.
  This wrecks real-time performance, so routers would not do this.
  They copy the contents of flash to RAM as part of booting.

Normal PCs:

- have used a battery-backed-up CMOS RAM module for firmware settings.
  Everything is CMOS now, I think, but when this dedicated memory was
  added to the PC, CMOS was notable.  Especially for its low standby
  power requirement.

- use flash memories with sequential access for firmware (SPI).  The
  power-on sequence fetches the firmware and dumps it into RAM so that
  it can be accessed randomly.  Dedicated programs are needed to
  update the flash through the SPI interface.

- uses RAM, just as we are used to

- uses SSD, with its own controller to emulate HDD but more quickly.