Bit Torrent, Updating Linux

[Lennart Sorensen]

On Thu, Jul 09, 2009 at 03:26:07PM -0400, Rajinder Yadav wrote:
> I am trying to understand what you're sharing, and with my limited
> knowledge I can only imagine how things (should) work from a higher
> level.
> 
> About wasted space, not quite sure I follow. If Package A was made
> download-able i.e. gcc4.3, it would have to reside on a server and one
> or more mirror, which is how things are done today. If you introduce a
> bit torrent (BT), where does the wasted space come from?

Bittorrent is only useful if lots of users participate.  To participate
they have to have the files to share with others.  That means they have
to keep the files after they download and install them.  If they don't
then they won't do any sharing as soon as they are down downloading and
installing them.

> Overall throughput of a BT download should be faster when connecting
> to several slow servers, if compared to ftp/http connected to a single
> slow server.

Not even close.  The server using ftp/http can use the kernel's sendfile
function and offload the whole job to the kernel.  bittorrent requires
the cpu to checksum each block and packetize it, which means entirely
userspace.  There is no server capable of saturating a 1gbit link with
bittorrent.  It is trivial to do so for ftp and http.

> If ftp/http downloads from a slow server or BT downloads how does that
> make the server load any different? I am not saying it does not, just
> doesn't seem to make sense =)

bittorrent has overhead, so it requires more bandwidth than just sending
the files would.

> I can see the issue of logistic to make sure all the mirror servers
> are always up to date with the main package server. Otherwise BT will
> not deliver on its throughput.
> 
> About a decent server being faster than BT. Would you know if it's the
> norm today that "most" (mirror) servers out there are decent? What
> would constitute decent in terms of kb/sec I am wondering, roughly the
> top speed of a typical home internet connection? If so than the
> argument for BT is mute.

bittorrent simply sucks as a distribution method.

Oh and most packages are so small that you wouldn't get any decent chunks
going at all which bittorrent also rellies on.

You are not the first to suggest it, but there is a reason no distribution
uses bittorrent for anything other than distributing iso images, and
even for that http and ftp are far supperior.

It doesn't help of course that ISPs hate bittorent in general.

You would be better off using the technique used by download accelerators,
which request byte ranges within a file using http (something supported
by some web servers), and then do multiple http transfers of the same
file of different ranges.  It could even be from different servers at
the same time.  This would work way better than bittorrent.  It is
compatible with proxy servers, ISPs don't hate it, it uses something the
server would already have, and avoids something the server doesn't want
to have.  It also works with sendfile and is hence offloadable to the
kernel by apache and the like.  Very efficient and fast.

So to download a 2MB package, you could request bytes 0-100k from server1,
100k-200k from server2, 200k-300k from server3, and when each chunk
finishes start another.  That way you might get 10 pieces from server1,
6 from server2 and the last 4 from server3, depending on their speeds
and loads.

Look up 'http byte serving'.

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists