[Simple-cdd-devel] Bug#861198: Shutting down public FTP services

[Enrico Zini]

On Thu, Apr 27, 2017 at 08:26:13AM -0700, Vagrant Cascadian wrote:

> I want to keep an eye to a patch that would be acceptible to the release
> team for stretch at this point, so maybe only refactoring this new code
> for a smaller, more readable diff would be best.

> Below is a crude implementation using python3-urllib3. Maybe it's good
> enough.

I would be tempted to rebase the download code on
urllib.request.urlretrieve, because it avoids an external dependency and
it does it all in one shot:
https://docs.python.org/3/library/urllib.request.html#urllib.request.urlretrieve

The only thing it does not do is verifying the Last-Modified header.
However, since we already have extrafiles, we could use the checksums in
there to see if the files need redownloading. That would avoid the
worries about parsing timestamps, too.

> It also supports using a proxy, although it seems to require a bit of
> code duplication. More elegant support for proxies would be nice.

The standard library's urllib is supposed to look at $http_proxy and use
it, according to https://docs.python.org/3/library/urllib.request.html
so at least in theory proxies are a non issues.

> Ironically, there are no remaining calls to wget in mirror_wget.py.
> Probably should deprecate and/or remove wget_debian_mirror and other
> variables that reference wget...

I would do that, leaving the variable so that nothing explodes on old
config files, but marking it as deprecated/ignored. I could even add an
"ignored" flag to variables that makes it throw an exception if the code
uses them.

Would you like me to try and provide a version which uses urlretrieve?


Enrico

-- 
GPG key: 4096R/634F4BD1E7AD5568 2009-05-08 Enrico Zini <enrico at enricozini.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/simple-cdd-devel/attachments/20170428/2be94104/attachment.sig>