[Pkg-bazaar-commits] r151 ./bzr-builddeb/people/jdw/merge_upstream: Merge upstream tarball fetching work.
James Westby
jw+debian at jameswestby.net
Tue Jul 10 21:04:28 UTC 2007
------------------------------------------------------------
revno: 151
committer: James Westby <jw+debian at jameswestby.net>
branch nick: merge_upstream
timestamp: Tue 2007-07-10 22:04:28 +0100
message:
Merge upstream tarball fetching work.
modified:
README
builder.py
debian/changelog
debian/control
specs/upstream-tarball-fetching
------------------------------------------------------------
revno: 132.1.2
merged: jw+debian at jameswestby.net-20070629171143-eutow6qejn4nwrfy
committer: James Westby <jw+debian at jameswestby.net>
branch nick: fetch_upstream2
timestamp: Fri 2007-06-29 18:11:43 +0100
message:
Update the spec for upstream-tarball-fetching to say it is implemented.
------------------------------------------------------------
revno: 132.1.1
merged: jw+debian at jameswestby.net-20070629170651-p0j61dgqkxsx7mua
committer: James Westby <jw+debian at jameswestby.net>
branch nick: fetch_upstream2
timestamp: Fri 2007-06-29 18:06:51 +0100
message:
Use uscan to retrieve upstream tarballs.
If a tarball is not found in the orig-dir and the user has a watch file then
use uscan to grab the tarball from upstream, and then repack it if needed
and place it in the orig-dir so that the build can continue.
If you are in merge mode then a dch -v will be all you need to try and build
a new upstream version.
This totally ignores signature checking and the like, but I am not sure how
I could handle that.
-------------- next part --------------
=== modified file 'README'
--- a/README 2007-07-10 21:02:32 +0000
+++ b/README 2007-07-10 21:04:28 +0000
@@ -278,7 +278,9 @@
$ mkdir tarballs
Then copy the ``.orig.tar.gz`` file for the current version in to this
-directory.
+directory. If you do not have the upstream tarball for the current version,
+but you do have a ``watch`` file detailing where it can be found then the
+plugin will automatically retrieve the tarballs as they are needed.
If upstream holds their code in a bzr branch then you don't need to create the
tarballs yourself, you can instead have the plugin export the upstream code and
@@ -336,7 +338,10 @@
To create a non-native package you need an upstream tarball to build against.
Set the ``orig-dir`` variable to the directory containing the tarball that
you want to use and the plugin will pick it up and you will have a non-native
-package.
+package. If you do not have the upstream tarball corresponding to the version
+of the package you are trying to build, but you have a ``watch`` file
+detailing where it can be found then it will be automatically retrieved when
+needed.
However sometimes you might be upstream of a package as well as Debian
maintainer, but it is not a native package. In that case you may version
=== modified file 'builder.py'
--- a/builder.py 2007-07-10 21:02:32 +0000
+++ b/builder.py 2007-07-10 21:04:28 +0000
@@ -93,6 +93,55 @@
if keep_source_dir:
raise NoSourceDirError;
+ def _watchfile_name(self):
+ watchfile = 'debian/watch'
+ if self._properties.larstiq():
+ watchfile = 'watch'
+ return watchfile
+
+ def _has_watch(self):
+ watchfile = self._watchfile_name()
+ if not self._tree.has_filename(watchfile):
+ info("There is no debian/watch file, so can't use that to"
+ " retrieve upstream tarball")
+ return False
+ if self._tree.path2id(watchfile) is None:
+ info("There is a debian/watch file, but it needs to be added to the "
+ "branch before I can use it to get the upstream tarball")
+ return False
+ return True
+
+ def _get_upstream_from_watch(self):
+ (tmp, tempfilename) = tempfile.mkstemp()
+ tmp = os.fdopen(tmp, 'wb')
+ watch_id = self._tree.path2id(self._watchfile_name())
+ assert watch_id is not None, "watchfile must be in the tree"
+ watch = self._tree.get_file_text(watch_id)
+ tmp.write(watch)
+ tmp.close()
+ info("Using uscan to look for the upstream tarball")
+ try:
+ r = os.system("uscan --upstream-version %s --force-download --rename "
+ "--package %s --watchfile %s --check-dirname-level 0" % \
+ (self._properties.upstream_version(),
+ self._properties.package(), tempfilename))
+ if r != 0:
+ raise DebianError("uscan failed to retrieve the upstream tarball")
+ finally:
+ os.unlink(tempfilename)
+ # Tarball is now renamed in the parent dir, either as .tar.gz or .tar.bz2
+ from repack_tarball import repack_tarball
+ fetched_tarball = os.path.join('..', self._tarball_name())
+ desired_tarball = self._tarball_name()
+ if not os.path.exists(fetched_tarball):
+ fetched_tarball = fetched_tarball[:-2] + 'bz2'
+ if not os.path.exists(fetched_tarball):
+ raise DebianError("Could not find the upstream tarball after uscan "
+ "downloaded it.")
+ repack_tarball(fetched_tarball, desired_tarball,
+ target_dir=self._properties.tarball_dir())
+ os.unlink(fetched_tarball)
+
def _find_tarball(self):
"""Find the upstream tarball and return it's location.
@@ -102,11 +151,16 @@
tarballdir = self._properties.tarball_dir()
tarball = os.path.join(tarballdir,self._tarball_name())
info("Looking for %s to use as upstream source", tarball)
- if not os.path.exists(tarballdir):
- raise DebianError('Could not find dir with upstream tarballs: '
- +tarballdir)
if not os.path.exists(tarball):
- raise DebianError('Could not find upstream tarball at '+tarball)
+ if not self._has_watch():
+ raise DebianError('Could not find upstream tarball at '+tarball)
+ else:
+ if not os.path.exists(tarballdir):
+ os.mkdir(tarballdir)
+ else:
+ if not os.path.isdir(tarballdir):
+ raise DebianError('%s is not a directory.' % tarballdir)
+ self._get_upstream_from_watch()
return tarball
def _tarball_name(self):
=== modified file 'debian/changelog'
--- a/debian/changelog 2007-07-10 19:15:44 +0000
+++ b/debian/changelog 2007-07-10 21:04:28 +0000
@@ -1,10 +1,14 @@
-bzr-builddeb (0.17.1) unstable; urgency=low
+bzr-builddeb (0.18) unstable; urgency=low
* Remove any 'debian/' directory from the upstream tarball in merge mode,
so that the branch contains all of the files that will appear there.
* Allow the changelog entries to have no author information.
* Add a working-tree option to the configuration files that if set always
builds the working tree rather than the last revision in the branch.
+ * uscan is now used to download an upstream tarball if the version required
+ is not available and the user has a watch file. Thanks to Daniel Holbach
+ for the insipration for how to implement this. This makes devscripts a
+ dependency.
-- James Westby <jw+debian at jameswestby.net> Tue, 10 Jul 2007 20:15:13 +0100
=== modified file 'debian/control'
--- a/debian/control 2007-07-10 20:52:45 +0000
+++ b/debian/control 2007-07-10 21:04:28 +0000
@@ -10,7 +10,7 @@
Package: bzr-builddeb
Architecture: all
-Depends: bzr (>= 0.18~), python-debian (>= 0.1.3), ${python:Depends}, python-deb822 (>= 0.3), dpkg-dev, fakeroot, bzrtools (>= 0.18)
+Depends: bzr (>= 0.18~), python-debian (>= 0.1.3), ${python:Depends}, python-deb822 (>= 0.3), dpkg-dev, fakeroot, bzrtools (>= 0.18), devscripts
Provides: bzr-buildpackage
XB-Python-Version: ${python:Versions}
Description: bzr plugin for Debian package management
=== modified file 'specs/upstream-tarball-fetching'
--- a/specs/upstream-tarball-fetching 2007-01-31 21:44:25 +0000
+++ b/specs/upstream-tarball-fetching 2007-06-29 17:11:43 +0000
@@ -1,7 +1,8 @@
Fetching the upstream tarball from some other location
------------------------------------------------------
-Status: Draft
+Status: Implemented
+Version: 0.18
Rationale
=========
@@ -20,57 +21,8 @@
Design
======
-A new config variable
-
-tarball-fetch-location
-
-that takes a URI to a directory that upstream uses for releases on a server.
-
-When building a package it will look for the tarball in orig-dir. If it is
-not found it will assume that it is a new release, and try and download the
-tarball from tarball-fetch-location. If this fails then it will error out
-as usual. If it suceeds the tarball will be placed in the build area. Also
-the plugin will try and create orig-dir if it doesn't already exist, and try
-and put it in there. This will save downloading each time (and the reason
-for not supporting remote orig-dir). If it can't put the file there it will
-just mention that and carry on, as we can assume that the user doesn't want
-the cacheing.
-
-Unsolved Problems
-=================
-
-There are two problems I envisage
-
-1. The upstream provides a different format tarball.
-
-This could be done later, but it would be easy to add auto-repacking of
-at least .bz2 tarballs. However the naming differences lead to point 2.
-
-2. The upstream has a different tarball name.
-
-Debian has strict rules on the name of the tarball. Upstream wont follow this.
-This means that builddeb needs to be able to infer the upstream tarball
-name from the debian name (or the information).
-
-There are a few alternatives as I see it.
-
- * Require the upstream name.
-
- Broken as it removes the generality of the solution.
-
- * Require a mapping from debian version etc. to upstream name.
-
- This seems best. The implementation needs some thought, as it needs to
- be flexible enough for correct specification but not overly complicated.
- For instance uscan already has this sort of thing, so perhaps use that,
- or even just parse debian/watch if present.
-
-Code changes
-============
-
-Add support for the option.
-Add command line option.
-Add transports for at least HTTP and FTP.
-Add name mangling support.
-Change the error message for no tarball in orig-dir to mention this option.
+If a watch file is included in the package, and the tarball is not found then
+uscan is asked to retrieve the desired version. It is then repacked if
+necessary and placed in the orig-dir so the build can be done, and the
+download wont have to be done next time.
More information about the Pkg-bazaar-commits
mailing list