[Apt-cacher-ng-users] Using acng to serve reprepro prevents successful download

Tue Aug 29 20:37:05 UTC 2017

Hello Eduard,

I hope I'm posting in the right place, my issue comes with too many
configuration caveats for a full bug-report (sorry for the long logs
below). I will try and narrow them down with further testing, but so
far it's reliably affected by the DontCacheResolved config option. I'm
running on stretch armhf as close to the defaults as possible. I've
attached my full puppet configuration in case that's easier for you,
but to summarise my settings:

---
/etc/apt-cacher-ng/reprepro.conf
DontCacheResolved: emorrp1.name
LocalDirs: games /srv/reprepro/games

* apt-cacher-ng 2-2, reprepro 5.1.1-1, apt 1.4.7
* /var/cache/apt-cacher-ng and /srv/reprepro mounted from lvm
partition on external harddrive
* groups for user apt-cacher-ng : apt-cacher-ng reprepro
* a single, signed, reprepro repo "games" for distribution "stretch"
available in amd64 and i386
---

On the client, I have Acquire::http::proxy "http://debian. (subdomain
of my email address) :3142/" and a source line of "deb http://debian
... :3142/games stretch main". I am able to complete an "apt update"
without any problems, so it's able to download at least the InRelease
file and pick up the gpg key etc. There's also no issue with external
repos, I've installed packages, upgraded etc. Unfortunately, I can't
complete a deb download and it appears to have something to do with
byte sizes:

$ sudo apt install --reinstall baldurs-gate-1-en-data
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 23 not upgraded.
Need to get 1,860 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://debian ... :3142/games stretch/main amd64
baldurs-gate-1-en-data all 49.1 [1,860 MB]
Get:1 http://debian ... :3142/games stretch/main amd64
baldurs-gate-1-en-data all 49.1 [1,860 MB]
Get:1 http://debian ... :3142/games stretch/main amd64
baldurs-gate-1-en-data all 49.1 [1,860 MB]
80% [1 baldurs-gate-1-en-data 1,860 MB/1,860 MB 100%]
                               3,866 kB/s 7min 48s^C

/var/log/apt-cacher-ng/apt-cacher.log
1504018593|I|116712452|86.169.17.164|debian ...
/games/pool/main/b/baldurs-gate-1-en-data/baldurs-gate-1-en-data_49.1_all.deb
1504018593|O|116285101|86.169.17.164|debian ...
/games/pool/main/b/baldurs-gate-1-en-data/baldurs-gate-1-en-data_49.1_all.deb
1504018714|I|858|86.169.17.164|debrep/dists/stretch/InRelease [HTTP
error, code: 404]
1504018714|E|217|86.169.17.164|debrep/dists/stretch/InRelease [HTTP
error, code: 404]
1504018715|O|228|86.169.17.164|debrep/dists/stretch-updates/InRelease
1504018717|O|230|86.169.17.164|debrep/dists/stretch-backports/InRelease
1504018717|I|63388|86.169.17.164|security.debian.org/dists/stretch/updates/InRelease
1504018717|O|63207|86.169.17.164|security.debian.org/dists/stretch/updates/InRelease
1504018717|O|218|86.169.17.164|debrep/dists/stretch/Release
1504019324|E|1860072232|86.169.17.164| [HTTP error, code: 200]
1504019923|E|1860072232|86.169.17.164| [HTTP error, code: 200]

(size and sha512sum on server)
1860072012 /srv/reprepro/games/pool/main/b/baldurs-gate-1-en-data/baldurs-gate-1-en-data_49.1_all.deb
afb629479d85fd697cdd6509fffc3e8e9d7e104d268515bbbc42aa30bc4646e1e3e6cce0e94f1d7d42d5c63ee5131ad47b2e89b68d94b2016c6914d176261ceb
baldurs-gate-1-en-data_49.1_all.deb

If I disable the "DontCacheResolved", apt download and install
completes successfully, alternatively I can manually download it in my
browser from http://debian:3142/games/pool/main/b/baldurs-gate-1-en-data/baldurs-gate-1-en-data_49.1_all.deb
with a sha512sum which matches the on-disk file. While I've been
typing this, I've tried another download with wget which may or may
not shed some light given it fails at the same byte mark:

$ wget http://debian ... :3142/debian ...
:3142/games/pool/main/b/baldurs-gate-1-en-data/baldurs-gate-1-en-data_49.1_all.deb
--2017-08-29 20:06:48--  http://debian ... :3142/debian ...
:3142/games/pool/main/b/baldurs-gate-1-en-data/baldurs-gate-1-en-data_49.1_all.deb
Resolving debian ... (debian ...)... 86.169.17.164
Connecting to debian ... (debian ...)|86.169.17.164|:3142... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1860072012 (1.7G) [application/x-debian-package]
Saving to: ‘baldurs-gate-1-en-data_49.1_all.deb.2’

baldurs-gate-1-en-data_49.1_all.deb.2
99%[====================================================================================================================>
]   1.73G  --.-KB/s    in 24m 13s

2017-08-29 20:31:02 (1.22 MB/s) - Read error at byte
1859945580/1860072012 (Connection timed out). Retrying.

--2017-08-29 20:31:03--  (try: 2)  http://debian ... :3142/debian ...
:3142/games/pool/main/b/baldurs-gate-1-en-data/baldurs-gate-1-en-data_49.1_all.deb
Connecting to debian ... (debian ...)|86.169.17.164|:3142... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1860072012 (1.7G) [application/x-debian-package]
Saving to: ‘baldurs-gate-1-en-data_49.1_all.deb.2’

baldurs-gate-1-en-data_49.1_all.deb.2
99%[====================================================================================================================>
]   1.73G  --.-KB/s    in 23m 38s

2017-08-29 20:54:41 (1.25 MB/s) - Read error at byte
1859945580/1860072012 (Connection timed out). Retrying.

(size and reliable sha512sum on client)
1859945580 baldurs-gate-1-en-data_49.1_all.deb.2
a6d2df586bd7cc1ffe7e1897ffbafa5669ba9c291aceefa744ce5ff54c539ecbced15b25e37e15b658bd8d7ac70561be74739ab441bd142f714049ddf67d9989
 baldurs-gate-1-en-data_49.1_all.deb.2

Possible external factors:
* raspberry pi 3 hardware
* data on external harddrive
* client connection over wifi
* large 1.8G file (relative to ram/buffers etc.)
* configuration via puppet
* original debootstrap image wrapped by "rpi23-gen-image"
--
Phil Morrell
-------------- next part --------------
A non-text attachment was scrubbed...
Name: apt.pp
Type: application/octet-stream
Size: 1448 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/apt-cacher-ng-users/attachments/20170829/e2333360/attachment.obj>