Bug#804239: [uscan] Failure when basename is in query string

[Osamu Aoki]

Hi,

On Fri, Nov 06, 2015 at 01:48:49PM +0000, Robie Basak wrote:
> Package: devscripts
> Version: 2.15.8
> 
> If the download URL is a query string, such as
> http://www.example.com/download?file=package-version.tar.gz, then uscan
> ends up downloading the file as package-version.download which
> subsequently causes mk-origtargz to fail with something like:
> 
> Parameter ../package-version.download does not look like a tar archive or a zip file. at /usr/bin/mk-origtargz line 330.
> 
> As far as I can tell, this makes uscan unusable for me for mysql-5.6.
> More specifics for my case:
> 
> http://dev.mysql.com/downloads/mysql/5.6.html?os=src lists opaque link
> URLs for the actual upstream tarballs (eg.
> http://dev.mysql.com/downloads/file/?id=459308) but does have links with
> versions embedded for the gpg files (eg.
> http://dev.mysql.com/downloads/gpg/?file=mysql-5.6.27.tar.gz) and I know
> how to construct a link for a corresponding release tarball (eg.
> http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.27.tar.gz).
> 
> So it appeared to me that I could coax uscan into working with this
> scheme and came up with the following:
> 
> opts=downloadurlmangle=s/\/downloads\/gpg\/?\?file=(mysql-([\d\.]*).tar.gz)/\/get\/Downloads\/MySQL-5.6\/$1/,pgpsigurlmangle=s/\/get\/Downloads\/MySQL-5.6\/(mysql-([\d\.]*).tar.gz)/\/downloads\/gpg\/?file=$1/ \
> http://dev.mysql.com/downloads/mysql/5.6.html?os=src /downloads/gpg/\?file=mysql-([\d\.]*).tar.gz
> 
> This works to find the available versions and does download and gpg verify
> the release tarball correctly. However it then fails with:
> 
> Parameter ../mysql-5.6-5.6.27.download does not look like a tar archive or a zip file. at /usr/bin/mk-origtargz line 330.
> uscan: error: mk-origtargz --package mysql-5.6 --version 5.6.27 --compression gzip --directory .. --copyright-file debian/copyright ../mysql-5.6-5.6.27.download gave error exit status 255
> 
> AFAICT the cause is the following snippet in uscan:
> 
>     # Remove HTTP header trash
>     if ($site =~ m%^https?://%) {
>         $newfile_base =~ s/\?.*$//;
>         # just in case this leaves us with nothing
>         if ($newfile_base eq '') {
>             $newfile_base = "$pkg-$newversion.download";
>         }
>     }

This is ugly workaround which may be better to be replaced with error if
there is no filenamemangle rule defined to make $newfile_base

> It seems to me that the "just in case" handling will always cause a
> future failure because mk-origtargz will always reject this name. I
> haven't tested this but if instead of .download it used .tar.gz then it
> would work in my case.
> 
> I haven't found any way of working around this but suggestions
> appreciated. In the meantime I'm filing this bug because:

Did you try filenamemangle which can generate filename from the full
href string.

       filenamemangle=rules
           Normalize the downloaded tarball filename string
           <upkg>-<uversion>.tar.gz from the selected href string.

           If this option doesn't exist, the default upstream tarball filename
           is found by taking the last component of the URL and removing
           everything after any '?'

(Yes, I should install release version of uscan to test this by my self
... but your help to check this is most appreciated.  I think this part
of code is problematic.)

At least, warning should be printed out by uscan.  I will work on
together with #526450 and #803948.

Osamu


Osamu