[Debtorrent-devel] The Next Steps: Apt Interaction

[Cameron Dale]

After the recent release, the next step in the process is a further integration
with apt to download from archives. From the wiki:

    Implement proxy functionality to receive input from apt about which
    packages to download.

I've been thinking about this step, and I now believe it will be the most
complicated of the initial steps in this project. Here is the functionality I
would like to have when this step is complete:

 * listen on a port for HTTP requests from apt (already complete)
   - requests will be of the form:
       http://localhost:port/www.mirror.com/path/to/file

 * respond to requests for Packages* files:
   - download it from the specified mirror
   - pass the downloaded file to apt
   - create a download directory based on the mirror name, path and file name
   - save the (uncompressed) file in the download directory
   - start a debtorrent download for the torrent of the Packages file

 * respond to requests for Packages.diff directory:
    - return 404 error, file not found, for now (so apt requests the full
      Packages file)
    - later can use these to reduce Packages download bandwidth

 * respond to requests for *.deb files by:
   - enable the file in any debtorrent downloads that have it
   - wait for the file to download
     + the debtorrent downloader may use HTTP to download the file from the
       mirror if the torrent download is taking too long or it is unable to
       find peers to download it from (rare/unseeded files)
   - pass the downloaded file to apt

 * respond to requests for any other file (Release, Release.gpg, Sources*,
   *.dsc, *.diff.gz, *.tar.gz, etc...):
   - download it from the mirror specified
   - pass the downloaded file to apt

This sounds like a lot of work, but it is simplified by some of the
functionality already being present in BitTornado. I have reduced it to the
following (simpler) steps:

1. make sure the current HTTP downloader works with mirrors (it might already)
2. start setting the 'http-seeds' info for torrents
3. make the debtorrent downloader use the http-seeds info as a backup download
4. make the apt listener use the http downloader for all requested files
5. modify btlaunchmany to start new torrents when Packages files are downloaded
6. modify btlaunchmany to enable files when .deb files are requested
7. make btlaunchmany wait for requested .deb files to download

Hopefully that's clear, and has captured most of the functionality
needed to interact with apt. If I've missed anything please let me
know.

Cameron