Bug#884604: devscripts: [uscan] please support hexadecimal encoded URLs

[Osamu Aoki]

Hi,

This wishlist bug seems to be a low hanging tasty fruit.

On Sun, Dec 17, 2017 at 04:05:20PM +0100, Michael Banck wrote:
> Package: devscripts
> Version: 2.15.3+deb8u1
> Severity: wishlist
> 
> Dear Maintainer,
> 
> I have the issue that one of my upstreams is hosted on a GForge
> instance, and its download urls are (needlessly I guess) encoding the
> filenames in hexadecimal, e.g.:
> 
> https://ccpforge.cse.rl.ac.uk/gf/download/frsrelease/574/8924/%64%6c%5f%63%6c%61%73%73%5f%31%2e%31%30%2e%74%61%72%2e%67%7a
> 
> The part after 8924/ translates to "dl_class_1.10.tar.gz" which I would
> need to furhter mangle to "dl-poly-classic-1.10.tar.gz", but that's
> another matter.
> 
> I managed to find the correct line (out of many) by doing a pagemangle
> on the link text (which contains the filename) and then mangling the URL
> back via downloadurlmangle, 

Before downloadurlmangle, uscan expect URL is not mangles in hex
encoding but in plain text.

> but uscan refuses to download the URL it
> correctly identified as candidate saying
> 
> uscan debug: https://ccpforge.cse.rl.ac.uk/gf/download/frsrelease/574/8924/%64%6c%5f%63%6c%61%73%73%5f%31%2e%31%30%2e%74%61%72%2e%67%7a by filenamemangle rule.
> uscan warn: Fix filenamemangle to produce a filename with the correct version
> 
> The current debian/watch is:
> 
> version=4
> opts="pagemangle=s/\">dl_class_([\d.]+)\.tar\.gz/dl_class_$1\.tar\.gz\">/,downloadurlmangle=s/dl_class_([\d.]+)\.tar\.gz//,filenamemangle=s/.*dl_class_([\d.]+)\.tar\.gz/dl-poly-classic-$1\.tar\.gz/"
> \
> https://ccpforge.cse.rl.ac.uk/gf/project/dl_poly_classic/frs/
> .*/gf/download/frsrelease/.*
> 
> As filenamemangle does not allow for the /e flag (and thus some decoding
> regexp like 's/%([A-Fa-f\d]{2})/chr hex $1/eg'), I can't do anything
> about the URL currently AFAICT.
> 
> So I guess what I would need is some option to tell uscan that the last
> part of the URL should be decoded (or maybe it could do that
> automatically), from which it could figure out the version.

Maybe add "downloadurldecode" option.

If it is present downloadurl is pre-processed with specified algorithm
before getting processed with downloadurldecode rule.

For now, let's implement "downloadurldecode=percent-encoding".

Osamu