[Apt-cacher-ng-users] Simultaneous retrieval not being cached?

[Eduard Bloch]

#include <hallo.h>
* Anshuman Aggarwal [Wed, Nov 23 2011, 05:35:41PM]:
> Thanks Eduard. 
>  Here is the information you were looking for:
> Debian Variant: Ubuntu 10.04 Lucid Lynx
> apt-cacher-ng: 0.4.6 (based on ubuntu version since apt-cacher-ng binary didn't seem to have an obvious way of reporting version).
> 
> The test you just described below, would not be apples to apples
> because the url used is exactly the same. If I'm not wrong one of the

Apples/apples? "same file at the same time", exactly what you wrote
before.  If you mean something different then please care to explain
what. Ideally, listing which files, at which moment, having which
client connection, etc. (hint: "apt-get --print-uris dist-upgrade" and
track HTTP headers, see manual for debug flags).

> big advantages of the caching is that even though we have debian
> mirrors the filename is sufficient to cache the .deb.

Oh, that's almost ironical. You talk about "Debian mirrors" while using
Ubuntu. Especially since Debian und Ubuntu often contain files with the
same name but different contents (recompiled from Debian versions).

Of course, there is a strategy to classify different sources in order to
consider them equal, sharing the same cache contents (and download stream
when needed). See
http://www.unix-ag.uni-kl.de/~bloch/acng/html/config-serv.html#repmap
for details.

Regards,
Eduard.

> Is there a reason why you say 'DO NOT DO THIS'? 
> 
> Regards,
> Anshuman
> 
> On 23-Nov-2011, at 3:39 AM, Eduard Bloch wrote:
> 
> > #include <hallo.h>
> > * Anshuman Aggarwal [Tue, Nov 22 2011, 03:49:14PM]:
> >> Hi,
> >> Thanks for apt-cacher-ng. Works well when the file has been retrieved
> >> completely.
> >> 
> >> However, if two or more clients are trying to request the same file at the
> >> same time (as can happen when people are upgrading in an office)…it appears
> > 
> > First: please report at least the version used there.
> > 
> >> that the partial file already downloaded is not being cached (possibly
> >> because the in-memory retrieved file has not been added to the database
> >> used for retrieval/filesystem)…I'm pretty sure this is happening as the
> >> download bandwidth gets halved then goes to a third when simultaneously
> >> upgrading packages…any chance someone who has looked at the code can quirky
> >> confirm this?
> >> Based on that it is either a bug or a feature request..
> > 
> > apt-cacher-ng was designed to explicitly support this usage pattern,
> > considering all rules of concurency.
> > 
> > In fact, I can not reproduce this issue... (DON'T DO THIS AT HOME)
> > 
> > for x in `seq 10` ; do ( curl -s http://localhost:3142/ftp.fr.debian.org/debian/pool/main/n/nexuiz-data/nexuiz-data_2.5.2-6_all.deb > /dev/null & ) ; done
> > 
> > and "iftop -B -i lo" shows me phantastic throughput, almost factor 10
> > compared to the progress of:
> > 
> > watch -n1 du /var/cache/apt-cacher-ng/debrep/pool/main/n/nexuiz-data/nexuiz-data_2.5.2-6_all.deb
> > 
> > Please check your log and make sure the files are resolved to the
> > same cache locations.
> > 
> > Regards,
> > Eduard.
> > -- 
> > <pasc> man
> > <pasc> the AMD64 camp is not helped by the list of people supporting it
> > <pasc> when nerode is on your side, you know you're doing something wrong
> 
> 

-- 
People who think that real programmers write code without bugs need to upgrade
to a human brain.
        http://yosefk.com/c++fqa/class.html#fqa-7.2