[Debtorrent-commits] r30 - /debtorrent/trunk/TODO

Wed May 2 22:32:28 UTC 2007

Author: camrdale-guest
Date: Wed May  2 22:32:28 2007
New Revision: 30

URL: http://svn.debian.org/wsvn/debtorrent/?sc=1&rev=30
Log:
Added more to the TODO list

Modified:
    debtorrent/trunk/TODO

Modified: debtorrent/trunk/TODO
URL: http://svn.debian.org/wsvn/debtorrent/debtorrent/trunk/TODO?rev=30&op=diff
==============================================================================

--- debtorrent/trunk/TODO (original)
+++ debtorrent/trunk/TODO Wed May  2 22:32:28 2007
@@ -12,7 +12,7 @@
 between them like it was previously. This may not be an issue after a maximum 
 piece size is introduced, though pre-allocation may still be necessary to serve 
 downloaded files while other downloads continue. Pre-allocation with priorities 
-enabled does not pre=allocate the entire archive.
+enabled does not pre-allocate the entire archive.
 
 
 Comments
@@ -29,3 +29,61 @@
 some pieces are very small, and some very large, this is no longer accurate. 
 This may not need fixing though, as piece sizes will become more uniform in 
 later versions, and these statistics may not be needed.
+
+
+Proposal for modifications to apt-ftparchive
+
+Modifications need to be made to the debian archive management program 
+apt-ftparchive to support some of the upcoming features of debtorrent. Primarily, 
+the large files in a Packages file need to be broken up into sub-pieces, and a 
+SHA1 hashes of the sub-pieces created. These would need to be cached, so as to 
+minimize the computation needed to generate a Packages file. Then, the tracker
+address needs to be added to Release files, along with any other needed metainfo. 
+Finally, large Packages files need to be broken down into smaller sub-pieces, and 
+SHA1 sums generated for the sub-pieces generated. These could be stored in the 
+Release file, or in separate files (Packages.pieces). For each of these, some 
+numbers are needed on the amount of data needed to be added to the files, and 
+how much computation will be required to generate it.
+
+
+Determine how to name the directory for downloaded files
+
+A policy is needed for determing what name to give the directory that debtorrent 
+downloads files into. Using the full Packages file name from /var/lib/apt/lists 
+means the directory will need to be changed/redownloaded if a mirror is changed. 
+Using the origin field from the Release file could cause problems if another 
+archive uses the same origin, and means that multiple torrents will download to 
+the same directory. Something like "<origin>_<codename>_<section>_binary-<arch>" 
+might work. That will result in duplicate downloads, if someone stracks both
+sid and testing, for example. However, before downloading, the file could be 
+searched for in other directories, and hardlinked if found (and SHA1 sum 
+matches.
+
+
+Modify pre-allocation to not create directories
+
+Currently, the pre-allocation type of allocation will create all the 
+directories in the download, even if the files in the directory are not being 
+downloaded. This could be modified to create directories on demand to reduce 
+filesystem bloat.
+
+
+Add parsing of currently installed packages to determine what to download
+
+As a temporary measure, before full apt proxy support (dt://) is possible, 
+debtorrent could parse the currently installed packages list in 
+/var/lib/dpkg/status and use that list to determine which files to download. 
+Then, configuration options (installed-only, installed-and-updated, 
+updated-only) would tell debtorrent which files to download. updated-only would 
+check the new version with dpkg --compare-versions to determine if the new 
+version should be downloaded. The debtorrent directory could then be added as 
+a file:/ source to apt, and new packages to install would not be found and use 
+the backup method (http://...). Then, an upgrade path would be: 
+1) apt-get update
+2) run debtorrent with the new Packages files
+3) wait for the download of updated packages to complete
+4) run apt-get upgrade/dist-upgrade
+Or the process could be automated by a cron script that gives debtorrent new 
+Packages files every week/day/hour. Might need to parse the 
+/var/cache/apt/archive directory when run the first time to hardlink in files 
+that have already been downloaded.


