[Apt-cacher-ng-users] Synchronizing / keeping up to date with multiple repos in offline mode

[Eduard Bloch]

Hallo,
* Phil (list) [Fri, Sep 16 2016, 04:41:15PM]:
> I was wondering if anyone could point me in the right direction on how
> to do the following: I currently have apt-cacher-ng running with
> multiple repos on multiple architectures (i.e. debian 8/testing for
> i386/amd64 etc) and everything is working quite well as an online repo.
>  What I would like to do is run it in offline mode and only
> periodically update the repositories.  So my question is how to do this
> in as transparent a way as possible to the client machines?  The most
> common recommendation I have found for offline repository management is
> to use apt-offline but can that work (if so, how?) in conjunction with

IMHO apt-offline is just a fat wrapper script for apt-get commands with
an URL downloader built-in (one could use wget/curl/... and some
renaming tricks instead).

See https://help.ubuntu.com/community/AptGet/Offline/PrintUris for
the way it basically works, and there are more HOWTOs, like
http://askubuntu.com/questions/974/how-can-i-install-software-or-packages-without-internet-offline .
I vaguelly remember having seen first such instructions more than a
decade ago.

> apt-cacher-ng?  Is there any way to update apt-cacher based on the
> index/package files it already has cached or do I need to update per
> client/arch (via apt-offline) on the client side and then somehow merge
> in the updates to the cache server?  Has anyone done a post / wiki on
> this use case?

Your description lacks some details. Which host is in offline mode and
when? If apt-cacher-ng machine itself is going online for short time,
then the case is clear. Build your download list and just download
packages through it (probably throwing data away). Maybe with
apt-offline or like this:

apt-get --print-uris --force-yes -q dist-upgrade | cut -f2 -d"'" > dl.list
http_proxy=http://acnghost:acngport wget -i dl.list -O /dev/null

The tricky part here is how to deal with index data. apt-offline (or
the mentioned howtos) use simple tricks here, injecting previously
downloaded files into apt cache on the target machine, renaming to
mimick the apt's internal cache file name convention.
Which enables apt to run package set calculations for
update/install/remove/... , and in the next step the same trick is used
to inject package files.

For apt-cacher-ng the same trick does not work, because it uses
different cache format and stores extra meta data in separate files.
Although there is the special "import" function which does similar
things, it works differently - it uses file contents to build
fingerprints, and it needs (fresh) index files (at least the higher
level ones) present in the cache or updated once to match the
fingerprints. So it won't help in case of "pure offline" mode.

I think the most simple and most convenient solution for this usecase
would be if I introduce a new feature to acngtool. Maybe something like
this:

acngtool dl4cache LISTFILE storagefolder

Then you call it with a LISTFILE built with the command above (or from
apt-offline) on a networked machine and it produces a folder which can
be transfered to apt-cacher-ng machine and "installed" with a "cp -a"
command (i.e. simply copy its contents into the cache directory,
considering the file structure).

Does this sound good enough?

Networked machine could even be win32, although I am not doing regular
build tests with the Cygwin port... it's about time to check that
anyhow.

The problem with this approach would be the need to extract some
configuration information from the cache machine and bring it over to
the downloading machine. This probably needs a couple of tricks but
should be feasible.

Regards,
Eduard.