Spiders and crawlers

[Lennart Sorensen]

On Mon, Apr 05, 2010 at 04:51:34PM +0000, Christopher Browne wrote:
> If you're going down that road, why not barge ahead with APDB instead?
> 
> http://thedailywtf.com/Articles/Announcing-APDB-The-Worlds-Fastest-Database.aspx

Because it is features, reliability and functionally that sane people
look for.  Not speed.  Speed is nice, but way way down the list of things
to look for in a database.

> "Relational and post-relational databases have the notion of index
> look-up, which means that retrieving a piece of data involves a long,
> arduous process:
> 
> - Find, on disk, where the appropriate index file is stored
> - Look for a free block of RAM in which the index file can be loaded
> - Load the index file into memory
> - Iterate over each index entry until the desired key matches the index key
> - When/if found, load the actual data location into memory
> - Find, on disk, where the actual data is stored using the index locator
> - Look for a free block of RAM in which the actual data can be loaded
> - Load the actual data into memory

Ehm, OK, anyone that thinks CHS is a useful index these days has no
clue what they are talking about.  Also javascript is used client side.
I don't want the client touching the database directly.  So again, what
makes anyone think this person has any clue what they are talking about?

Apparently some people believe that the operating system is in the way
of getting things done.  Too bad it actualyl does a lot of useful work
that you don't want to have to implement in every application.

> With APDB, this process is reduced to one step: go to the actual
> location that matches the index key specified. No middleman, index
> files, or other nonsense needed; just go directly to the data you
> want."
> 
> Note that for those that are unhappy at how long it takes to access
> (possibly cached?) data via fread(), APDB has a further performance
> optimization.  Careful analysis showed that all modern disk drives use
> microcontrollers from just 2 manufacturers, who allow firmware
> rewrites, thereby supporting the following:
> 
> "By rewriting the harddrive’s firmware, APDB can operate in the most
> performant mode possible"

Speed really isn't that important (which is in my opinion what has made so
many people use mysql out of sheer ignorance).  Having it fast when only
one person uses it really doesn't matter.  Always doing the right thing,
and maintaining good performance when lots of people are using it matter
a lot more.  Being compliant with common standards so you can easily
move to another implementation someday is also nice.  postgresql and
some others are very SQL standards compliant.  Mysql certainly never
has been that I have seen.

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists