[gopher] Spidering teh gopherspace

[Cameron Kaiser]

> Anyway, on to a few questions: What kind of spidering rate would the 
> admins here accept? The spider will index types 0 and 1 (text documents 
> and menus) and currently does three hits per second (actually, a hit and 
> a 1/3 second delay). I think that's too fast - so how does a hit per 
> second sound like? I'll take forever to spider things, but at least it 
> wouldn't kill anyones server....

V-2 uses five seconds, actually, and it only grabs menus (in keeping with
the original Veronica concept), so it is a pretty low impact spider.

-- 
------------------------------------ personal: http://www.cameronkaiser.com/ --
  Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser at floodgap.com
-- Artificial intelligence is no match for natural idiocy. --------------------