Bug#685787: devscripts: Enabling uscan to simply remove files from upstream source
Andreas Tille
tille at debian.org
Sun Aug 26 13:30:44 UTC 2012
Hi Gregor,
I reverted one single line in your patch because this would break one of
my test cases namely
Vcs-Svn: svn://svn.debian.org/debian-med/trunk/packages/saint/trunk/
and does not fit my intention. You tried to check whether upstream might
contain a dir /^$pkg\w*$newversion$excludesuffix\.orig$/ which is most
probably never the case (why should upstream name something ".orig"?).
The intention of my check was to verify whether upstream source contains
something like a reasonable directory containing
/^$pkg\w*$newversion$/
and do not try to undirty the tarball if this is the case. It is true
that in this specific case we fail to name the directory ".orig" ...
but that's IMHO something we could hardly control.
I attached another Git formated patch and repeat my question whether
it might make sense to directly commit to the devscripts repository.
Kind regards
Andreas.
On Sat, Aug 25, 2012 at 02:59:33PM +0200, gregor herrmann wrote:
> On Fri, 24 Aug 2012 16:38:13 +0200, Andreas Tille wrote:
>
> > in a (bit longish) thread on debian-devel at l.d.o[1] there was some
> > discussion about enabling uscan to remove files from upstream archives
> > according to some information given in some control file. There was no
> > real consensus about what control file to use. The implementation below
> > is based on using debian/copyright but is easy to switch to other files
> > in case some other consensus might be reached.
>
> Thanks!
>
> > The attached patch does the following:
>
> Attached are some patches based on devscipts+git plus your original
> patch.
>
> The important one is
> 0003-Rename-directory-in-renamed-tarball-to-pkg-newversio.patch which
> renames the directory in the new tarball to
> $pkg - $newversion $excludesuffix .orig
> as recommended in DevRef 6.7.8.2.
>
>
> Cheers,
> gregor
>
>
> --
> .''`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
> : :' : Debian GNU/Linux user, admin, and developer - http://www.debian.org/
> `. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
> `- NP: Ry Cooder: Let's Work Together
> From 34d41b8480c3467304383df3633c03ce6c5f34e2 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gregoa at debian.org>
> Date: Sat, 25 Aug 2012 14:12:10 +0200
> Subject: [PATCH 1/5] whitespace
>
> ---
> scripts/uscan.pl | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index e118142..1ea4489 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -2099,7 +2099,7 @@ sub get_main_source_dir($$$) {
> print "-- Dirty tarball found.\n" if $verbose;
> if ( $main_source_dir ) { # if tarball is dirty but does contain a $pkg-$newversion dir we will not undirty but leave it as is
> print "-- No idea how to create proper tarball structure - leaving as is.\n" if $verbose;
> - return $tempdir;
> + return $tempdir;
> }
> print "-- Move files to subdirectory $pkg-$newversion.\n" if $verbose;
> $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion ;
> @@ -2108,7 +2108,7 @@ sub get_main_source_dir($$$) {
> unless ($file =~ /^\.\.?/) {
> # move("${tempdir}/$file", $main_source_dir) or die("Unable to move ${tempdir}/$file directory $main_source_dir\n");
> unless ( move("${tempdir}/$file", $main_source_dir) ) {
> - # HELP: why can't perl move not move directories????
> + # HELP: why can't perl move not move directories????
> print "Perl move seems to be not able to ` move(\"${tempdir}/$file\", $main_source_dir) ` ... trying system mv\n" if $debug;
> system( "mv ${tempdir}/$file $main_source_dir" ) ;
> }
> --
> 1.7.10.4
>
> From 67b13a72c801fb7419254784e443a90506819bc4 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gregoa at debian.org>
> Date: Sat, 25 Aug 2012 14:19:48 +0200
> Subject: [PATCH 2/5] s/fuffix/suffix/g
>
> ---
> scripts/uscan.pl | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index 1ea4489..cd69217 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -1483,9 +1483,9 @@ EOF
> if ( $nfiles_before == $nfiles_after ) {
> print "-- Source tree remains identical - no need for repacking.\n" if $verbose;
> } else {
> - my $excludefuffix = '+dfsg' ;
> + my $excludesuffix = '+dfsg' ;
> my $suffix = 'gz' ;
> - my $newfile_base_dfsg = "${pkg}_${newversion}${excludefuffix}.orig.tar.$suffix" ;
> + my $newfile_base_dfsg = "${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix" ;
> system("cd $tempdir; GZIP='-n -9' tar --owner=root --group=root --mode=a+rX -czf \"$absdestdir/$newfile_base_dfsg\" $globpattern") == 0
> or die("Excluding files failed (could not create tarball)\n");
> $symlink = 'no' # prevent symlinking or renaming
> --
> 1.7.10.4
>
> From d40d483fb870b1a8efea2b2c77a70cf3358f3fb5 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gregoa at debian.org>
> Date: Sat, 25 Aug 2012 14:29:50 +0200
> Subject: [PATCH 3/5] Rename directory in renamed tarball to $pkg -
> $newversion $excludesuffix .orig
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> DevRef § 6.7.8.2
> ---
> scripts/uscan.pl | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index cd69217..cb29f44 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -75,7 +75,7 @@ sub dehs_die ($);
> sub dehs_output ();
> sub quoted_regex_replace ($);
> sub safe_replace ($$);
> -sub get_main_source_dir($$$);
> +sub get_main_source_dir($$$$);
>
> sub usage {
> print <<"EOF";
> @@ -1465,7 +1465,8 @@ EOF
> system('unzip', '-q', '-a', '-d', $tempdir, "$destdir/$newfile_base") == 0
> or die("Repacking from zip to tar.gz failed (could not unzip)\n");
> }
> - my $main_source_dir = get_main_source_dir($tempdir, $pkg, $newversion);
> + my $excludesuffix = '+dfsg' ;
> + my $main_source_dir = get_main_source_dir($tempdir, $pkg, $newversion, $excludesuffix);
> unless ( -d $main_source_dir ) {
> print STDERR "Error: $main_source_dir is no directory";
> }
> @@ -1483,7 +1484,6 @@ EOF
> if ( $nfiles_before == $nfiles_after ) {
> print "-- Source tree remains identical - no need for repacking.\n" if $verbose;
> } else {
> - my $excludesuffix = '+dfsg' ;
> my $suffix = 'gz' ;
> my $newfile_base_dfsg = "${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix" ;
> system("cd $tempdir; GZIP='-n -9' tar --owner=root --group=root --mode=a+rX -czf \"$absdestdir/$newfile_base_dfsg\" $globpattern") == 0
> @@ -2070,8 +2070,8 @@ sub safe_replace($$) {
> }
> }
>
> -sub get_main_source_dir($$$) {
> - my ($tempdir, $pkg, $newversion) = @_;
> +sub get_main_source_dir($$$$) {
> + my ($tempdir, $pkg, $newversion, $excludesuffix) = @_;
> my $fcount = 0;
> my $main_source_dir = '' ;
> my $any_dir = '' ;
> @@ -2083,7 +2083,7 @@ sub get_main_source_dir($$$) {
> $fcount++;
> if ( -d $tempdir.'/'.$file ) {
> $any_dir = $tempdir . '/' . $file ;
> - $main_source_dir = $any_dir if ( $file =~ /^$pkg\w*$newversion$/i ) ;
> + $main_source_dir = $any_dir if ( $file =~ /^$pkg\w*$newversion$excludesuffix\.orig$/i ) ;
> }
> }
> }
> @@ -2092,7 +2092,7 @@ sub get_main_source_dir($$$) {
> }
> if ( $fcount == 1 and $any_dir ) {
> # Unusual base dir in tarball - should be rather something like ${pkg}-${newversion}
> - $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion ;
> + $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion . $excludesuffix . '.orig';
> move($any_dir, $main_source_dir) or die("Unable to move $any_dir directory $main_source_dir\n");
> return $main_source_dir ;
> }
> @@ -2102,7 +2102,7 @@ sub get_main_source_dir($$$) {
> return $tempdir;
> }
> print "-- Move files to subdirectory $pkg-$newversion.\n" if $verbose;
> - $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion ;
> + $main_source_dir = $tempdir . '/' . $pkg . '-' . $newversion . $excludesuffix . '.orig';
> mkdir($main_source_dir) or die("Unable to create temporary source directory $main_source_dir\n");
> foreach my $file (@files) {
> unless ($file =~ /^\.\.?/) {
> --
> 1.7.10.4
>
> From 482540af9ac9e8783a1ae85eac63965f60a2cbbd Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gregoa at debian.org>
> Date: Sat, 25 Aug 2012 14:41:18 +0200
> Subject: [PATCH 4/5] add support for .jar
>
> ---
> scripts/uscan.pl | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index cb29f44..0bab138 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -1410,7 +1410,7 @@ EOF
> $newfile_base = $newfile_base_gz;
> }
>
> - if ($repack and $newfile_base =~ /^(.*)\.zip$/) {
> + if ($repack and $newfile_base =~ /^(.*)\.(zip|jar)$/) {
> print "-- Repacking from zip to .tar.gz\n" if $verbose;
>
> system('command -v unzip >/dev/null 2>&1') >> 8 == 0
> @@ -1422,12 +1422,12 @@ EOF
> my $hidden = ".[!.]*";
> my $absdestdir = abs_path($destdir);
> system('unzip', '-q', '-a', '-d', $tempdir, "$destdir/$newfile_base") == 0
> - or die("Repacking from zip to tar.gz failed (could not unzip)\n");
> + or die("Repacking from zip or jar to tar.gz failed (could not unzip)\n");
> if (defined glob("$tempdir/$hidden")) {
> $globpattern .= " $hidden";
> }
> system("cd $tempdir; GZIP='-n -9' tar --owner=root --group=root --mode=a+rX -czf \"$absdestdir/$newfile_base_gz\" $globpattern") == 0
> - or die("Repacking from zip to tar.gz failed (could not create tarball)\n");
> + or die("Repacking from zip or jar to tar.gz failed (could not create tarball)\n");
> unlink "$destdir/$newfile_base";
> $newfile_base = $newfile_base_gz;
> }
> @@ -1463,7 +1463,7 @@ EOF
> system('command -v unzip >/dev/null 2>&1') >> 8 == 0
> or die("unzip binary not found. This would serve as fallback because tar just failed.\n");
> system('unzip', '-q', '-a', '-d', $tempdir, "$destdir/$newfile_base") == 0
> - or die("Repacking from zip to tar.gz failed (could not unzip)\n");
> + or die("Repacking from zip or jar to tar.gz failed (could not unzip)\n");
> }
> my $excludesuffix = '+dfsg' ;
> my $main_source_dir = get_main_source_dir($tempdir, $pkg, $newversion, $excludesuffix);
> --
> 1.7.10.4
>
> From f85b9c16db8e388cd6a62fbe8e8aabe9ef082c99 Mon Sep 17 00:00:00 2001
> From: gregor herrmann <gregoa at debian.org>
> Date: Sat, 25 Aug 2012 14:52:01 +0200
> Subject: [PATCH 5/5] Output messages about the files-excluding action.
>
> ---
> scripts/uscan.pl | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/uscan.pl b/scripts/uscan.pl
> index 0bab138..ba458ea 100755
> --- a/scripts/uscan.pl
> +++ b/scripts/uscan.pl
> @@ -1443,6 +1443,7 @@ EOF
> }
> }
>
> + my $excludesuffix = '+dfsg';
> if ( !$no_exclusion ) {
> my $data = Dpkg::Control::Hash->new();
> $data->load('debian/copyright');
> @@ -1465,7 +1466,6 @@ EOF
> system('unzip', '-q', '-a', '-d', $tempdir, "$destdir/$newfile_base") == 0
> or die("Repacking from zip or jar to tar.gz failed (could not unzip)\n");
> }
> - my $excludesuffix = '+dfsg' ;
> my $main_source_dir = get_main_source_dir($tempdir, $pkg, $newversion, $excludesuffix);
> unless ( -d $main_source_dir ) {
> print STDERR "Error: $main_source_dir is no directory";
> @@ -1488,7 +1488,7 @@ EOF
> my $newfile_base_dfsg = "${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix" ;
> system("cd $tempdir; GZIP='-n -9' tar --owner=root --group=root --mode=a+rX -czf \"$absdestdir/$newfile_base_dfsg\" $globpattern") == 0
> or die("Excluding files failed (could not create tarball)\n");
> - $symlink = 'no' # prevent symlinking or renaming
> + $symlink = 'files-excluded' # prevent symlinking or renaming
> }
> }
> }
> @@ -1519,6 +1519,8 @@ EOF
> print " and symlinked $renamed_base to it\n";
> } elsif ($symlink eq 'rename') {
> print " and renamed it as $renamed_base\n";
> + } elsif ($symlink eq 'files-excluded') {
> + print " and removed files from it in ${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix\n";
> }
> } elsif ($dehs) {
> my $msg = "Successfully downloaded updated package $newfile_base";
> @@ -1527,6 +1529,8 @@ EOF
> $msg .= " and symlinked $renamed_base to it";
> } elsif ($symlink eq 'rename') {
> $msg .= " and renamed it as $renamed_base";
> + } elsif ($symlink eq 'files-excluded') {
> + $msg .= " and removed files from it in ${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix\n";
> } else {
> $dehs_tags{'target'} = $newfile_base;
> }
> @@ -1537,6 +1541,8 @@ EOF
> print " and symlinked $renamed_base to it\n";
> } elsif ($symlink eq 'rename') {
> print " and renamed it as $renamed_base\n";
> + } elsif ($symlink eq 'files-excluded') {
> + print " and removed files from it in ${pkg}_${newversion}${excludesuffix}.orig.tar.$suffix\n";
> }
> }
> last;
> --
> 1.7.10.4
>
--
http://fam-tille.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Undo-one-part-of-Gregor-s-changes.-It-is-not-safe-to.patch
Type: text/x-diff
Size: 1119 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/devscripts-devel/attachments/20120826/9fb9ac50/attachment.patch>
More information about the devscripts-devel
mailing list