Bit Torrent, Updating Linux

Lennart Sorensen lsorense-1wCw9BSqJbv44Nm34jS7GywD8/FfD2ys at
Thu Jul 9 20:08:50 UTC 2009

On Thu, Jul 09, 2009 at 03:47:42PM -0400, Lennart Sorensen wrote:
> So to download a 2MB package, you could request bytes 0-100k from server1,
> 100k-200k from server2, 200k-300k from server3, and when each chunk
> finishes start another.  That way you might get 10 pieces from server1,
> 6 from server2 and the last 4 from server3, depending on their speeds
> and loads.
> Look up 'http byte serving'.

An example using the python-urlgrabber tool:

urlgrabber --range=,10000 package.1
urlgrabber --range=10000,20000 package.2
urlgrabber --range=20000,30000 package.3
urlgrabber --range=30000, package.4

cat package.* > package

The 4 urlgrabbers could run in parallel, and when all done you have
the whole file.  Of course s smart download tool would do the byte
range requests itself and write it to one file directly using seeks
as appropriate.  The only requirement is that the http server support
byte range requests, which many do, since it is required to do resume

Len Sorensen
The Toronto Linux Users Group.      Meetings:
TLUG requests: Linux topics, No HTML, wrap text below 80 columns

More information about the Legacy mailing list