[Apt-zip-devel] offline.debian.net method

[Giacomo A. Catenazzi]

Eddy Petrișor wrote:
> Giacomo A. Catenazzi wrote:
>> Eddy Petrișor wrote:

>> debian mirror are not authenticated.
> 
> I was talking about the .d.n server that does the assembly and 
> calculat.... (light bulb on)
> 
> OH, you mean the server will only do calculations and will leave 
> downloading to the connected machine? That would mean we would still 
> need some way to download stuff. Is that the javascript stuff you were 
> talking about?
> 
> So, let me get this straight:
> - disconnected machine (DM) uses another fetch method, let's call it 
> 'offline.d.n' and creates a script/set of files that allow the connected 
> machine (CM) to export the state of the DM to offline.debian.net
> - the CM will access offline.d.n and will upload the DM state
> - offline.d.n calculates dependencies and generates some javascript that 
> will get the necessary files for the DM to get it into the desired 
> state; offiline.d.n redirects (somehow) the CM to the page that has the JS
> - the JS script runs on the CM and, as a result, it starts downloading 
> the necessary files and placing them in a local archive
> - files are downloaded and ready to be brought back to the DM
> - on the DM, apt-zip-inst is ran
> 
>> And for my experiences, authentication is much more resource extensive
>> (captcha, confirmation via email [to the wrong people], ...).
> 
> I was saying that there should be some authentication on offiline.d.n to 
> prevent it being a target of DoS attacks via automatic uploads of 
> machine states.
> 
>> I think it would be simpler to limit bandwidth or number of downloads
>> (apache and lighttpd have such options).
> 
> now *I* am confused. Does offline.d.n does the downloads then the CM 
> gets the whole archive, or offline.d.n is only a dependency calculator?

/me too :-)

Javascript was one of my old proposal (without the server back-end),
but I didn't find real method to download multiple files.

I was thinking about:
- USER on DM: list of packages
- USER on CM: open o.d.n, upload the list
- SERVER: will find updated packages
- SERVER: build a tar and give the link to user
   (tar are cleaned after 12h)
- USER on CM: will click the link to download the
   tar
- USER will move the tar to DM and it will install it.

The dependencies calculation was in a "further step", so
very easy to program, but uses a more of bandwidth.

Your proposal (javascript) will optimize the resources,
looking also to the nearest mirror, so server side
will be very light. If we can do in this manner, it
will be great!


About authentication:
I don't think it is necessary, IMHO it requires to much
resources.  So, IMO, we start with the simple method without
authentication. If we find problems, we will add the authentication.

ciao
	cate


> 
>>> Also, apt-zip would be only responsible of creating a "local machine 
>>> status" snapshot (states of the packages+sources.list{,.d/*}).
>>>
>>> Still, I think this should *not* replace the current apt-zip-list and 
>>> apt-zip-inst scripts until we have something really functional.
>>
>> but also the fetch scripts should remain. They are very good for
>> automated job.
>>
>> I see the web version as an additional fetch method.
> 
> ok.
> 
>>> It depends on which packages are downloaded. We could impose a soft 
>>> limit of (say) 50MB for a apt-zip-server upgrade and request direct 
>>> usage of apt-zip-list update + apt-zip-list upgrade ... for bigger 
>>> downloads.
>>
>> let see. I really don't know usage pattern and how much people
>> will use it.
>> But if the method become popular, we would find easily mirror
>> machines.
>> Probably we should also talk with Ubuntu, to share resources.
> 
> Initially I thought offline.d.n does downloads itself (probably hosting 
> the service on a mirror will make this step fast), but that would ignore 
> the apt sources from the DM sources.list{,.d/*.sources.list} files and 
> would bottleneck the CM-offline.d.n connection or even trigger a DoS on 
> offline.d.n.
> 
> That's why I think is better to have all the downloading to the CM, but 
> the dependency calculation should be passed to offline.d.n.
> 
>