[Debtorrent-devel] Fwd: BitTorrent Protocol Expansion (Google SoC)

[Cameron Dale]

---------- Forwarded message ----------
From: Cameron Dale <camrdale at gmail.com>
Date: Apr 9, 2007 2:43 PM
Subject: Re: BitTorrent Protocol Expansion (Google SoC)
To: Anthony Towns <aj at azure.humbug.org.au>


Anthony Towns wrote:
>>> For the first half, ordering as:
>>>    1. will share all downloaded pieces with other interested clients
>>>     (already what bittornado does!)
>>>    2. implements variable size pieces for all packages
>>>     (use hacked up .torrent files that will get you an "interesting" bit
>>>      of the archive)
>>>    3. uses current Packages files as torrent files
>>>     (add Packages -> torrent parsing into bittornado so you don't need to
>>>      download duplicate information)
>>>    4. receives input from apt about which packages/pieces to download
>>>     (add separate scripts to parse /var/lib/dpkg/available and/or apt
>>>      to prioritise pieces?)
>>>    5. runs as a daemon, started from /etc/init.d
>>>     (automate it all)
>>> might work well -- that way it's possible to start releasing usable
>>> betas right from step (2).
>> Looks good to me. I'm not sure what you mean by "add separate
scripts to parse
>> /var/lib/dpkg/available and/or apt to prioritise pieces" though.
>
> Hrm, I meant /var/lib/dpkg/status which'll tell you what packages are
> currently installed; adding scripts to apt would be a pre-invoke script
> or similar, I guess.
>
> Sounds good.

It looks like we have some preliminary implementation steps to get started with.
 I'll add them to the wiki. I'm leaving off step 1 though as it's already done.

As it seems like the implementation is ready to be started, I was looking at the
first step to be sure we're not overlooking something that will bite us later.
Since the first step is variable size pieces, I'm wondering, are we sure this is
the way to go? What about my alternate suggestion on the wiki for a smart
ordering of pieces that would minimize the wasted bandwidth? Maybe we should
generate some statistics on this to check?

And what about the choice of BitTornado as a starting client to modify? I was
looking through the long bug list for it, and I see a couple of things that
might be a problem. First, it's UPnP support seems to be Windows based, and so
not functional. I looked around for other implementations, and it seems Twisted
has the only python-based UPnP implementation. Maybe we could use/borrow from
that? This might seem trivial, but given the trouble configuring
clients/firewalls to work together to get the most out of BitTorrent it might
become more important.

The second problem that comes up a lot with BitTornado is its complete lack of
support for encodings such as unicode and utf8. Do you foresee any problems with
the file names that we'll be dealing with? Is there some kind of standard
detailing the encoding that Debian archives use for their files?

I still think BitTornado is the way to go, I just want to make sure we have all
the information before deciding. I personally think the first problem can be
fixed later, and the second will not be a problem.

Cameron
-------------- next part --------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGGrN0Dx924g0gNq0RAgFzAJ9Btzz2A9ldoma6qMVj8PDpyqz6wQCfRTp8
YjS9FPOURuv83pON7TCbjUE=
=nlip
-----END PGP SIGNATURE-----