Bit Torrent, Updating Linux

[Lennart Sorensen]

On Thu, Jul 09, 2009 at 03:47:42PM -0400, Lennart Sorensen wrote:
> So to download a 2MB package, you could request bytes 0-100k from server1,
> 100k-200k from server2, 200k-300k from server3, and when each chunk
> finishes start another.  That way you might get 10 pieces from server1,
> 6 from server2 and the last 4 from server3, depending on their speeds
> and loads.
> 
> Look up 'http byte serving'.

An example using the python-urlgrabber tool:

urlgrabber --range=,10000 http://server1.org/package package.1
urlgrabber --range=10000,20000 http://server2.org/package package.2
urlgrabber --range=20000,30000 http://server3.org/package package.3
urlgrabber --range=30000, http://server4.org/package package.4

cat package.* > package

The 4 urlgrabbers could run in parallel, and when all done you have
the whole file.  Of course s smart download tool would do the byte
range requests itself and write it to one file directly using seeks
as appropriate.  The only requirement is that the http server support
byte range requests, which many do, since it is required to do resume
downloads.

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists