[Debtorrent-devel] Re: BitTorrent Protocol Expansion (Google SoC)

[Cameron Dale]

On 4/24/07, Anthony Towns <aj at azure.humbug.org.au> wrote:
> Okay, so I'll aim to catch you around 9am Wednesday my time, 4pm Tuesday
> your time.

Sounds good. Starting today I assume? I'll be waiting.

> If you're happy with a blog, that'll be great. I saw you'd started
> one previously, but hadn't gotten past a first post, so didn't want to
> presume.  If you do want to blog, we'll get it added to Planet Debian
> if you like.

Yeah, that was sort of the example blog that comes with the website
software (ikiwiki). I hadn't really planned on anyone reading it. ;) I
haven't been great with blogs in the past, but then I've never had
anything to blog about that I thought others would want to read.
Hopefully this will be different.

> > Are other students going to be emailing the list?
>
> That's the intention, I think.

Well, I guess I should too then.

> > I'm not sure what you mean by "Doing that either as part of the IRC/IM
> > chat"? I thought you meant updates I would do, for your and others
> > benefit. Now it looks like you mean updates we create together for the
> > benefit of others. Is that more what you had in mind?
>
> Well, if you can get them done by yourself for my and others' edification,
> that's brilliant; but if not, we should definitely make sure we get together
> and make some progress each week.

Sounds good.

> I see you've gone ahead and added the "debtorrent" spec to the wiki
> page. I presume that's a cut&paste from the link you left -- you probably
> should be careful to attribute the copyright and authorship of the
> original article properly in that case. Debian's a bit anal about such
> things. :)

Hmm, I'm not sure what to put then. I just updated it to change all
references from BitTorrent to DebTorrent. Since I've changed it, do I
need to mention their copyright of it? If so, how do I?

> One thought I had was that maybe a simple first step would be to generate
> a (variable piece size) torrent file from a Packages file, and convert
> bittornado to using that, before trying to make it use Packages files
> directly. Going through the protocol, it seems like there's not actually
> a lot that needs to be changed -- just the "piece length" field that
> needs to become a list of lengths, paralleling the "pieces" list that
> has the sha1sums.

Those were my thoughts as well.

> I've attached p2t.py which dumps a "torrent" file format based on a
> Packages file. It's ugly, hardcoded, and I imagine it's buggy too, but
> it might be a reasonable way to start getting to grips with making actual
> changes. I suspect there's going to be a lot of annoying (or hard...) work
> rearranging bittornado just to cope with variable-size pieces pretty soon.

Good work! Now I know who to come to if I get stuck with anything python. ;)

> That would make the (first pass) download process be:
>
>         * install debtorrent
>         * apt-get update
>         * run "p2t.py" to generate the debtorrent file you need
>         * run debtorrent to download the suite
>         * wait for the download to complete
>         * tell downloader to exit (it keeps uploading until this happens)

This depends on how many people you think are going to be testing an
early release such as this. I had actually thought that for early
testing we would have to create a debtorrent file (or a small number
of them) and distribute it with the debtorrent package, so that the
few number of initial testers will be in the same torrent(s) and not
stranded. This also allows us to setup the tracker and an initial seed
as well for the data.

By the way, when you say "install debtorrent", are you thinking this
will be a debian package, or a tarball? I have been thinking about
whether to bother packaging this as a .deb from the beginning
(requires python integration and some thought about file locations and
the such), or whether to stick with a tarball for the early testing.
On one hand, it would be pretty easy to start with the bittornado
packaging stuff and evolve it over time with the rest of the package,
but this could lead to unnecessary time wasted on something that is
not yet required. Your thoughts?

> Do you have any way in mind for emphasising the differences between the
> (evolving) debtorrent spec and the bittorrent spec? Bold, colours, some sort
> of comments or anything? Or just relying on the wiki diffs?

I had thought wiki diffs would be adequate, especially if the changes
comment mentions the new protocol version or something.

> Anyway, looks like there's lots of progress to report so far, and GSoC
> hasn't even officially started yet :)

Yep, the first update should be easy to write. :)

Cameron