Debtags and Debram (Was: Re: New debtags suite just uploaded)

[Enrico Zini]

--envbJBWh7q8WU6mo
Content-Type: text/plain; charset=iso-8859-15
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Jul 02, 2004 at 03:29:41PM +0000, Thaddeus H. Black wrote:

Thanks for the long post and history, and for taking the time to write
it down.  You could consider pasting most of it into a HISTORY file to
add to the debram package!

> I suppose
> that by now I have attained a better overall
> knowledge of the contents of the Debian archive
> than all but a very few people in the world,

Me too!  And you surely even better than me.
Doing this categorization work really improves the way of thinking and
understanding packages, and software in general.


> I did not know if you were serious.  How could I
> know?  Most people who post ideas on lists are
> not serious.  Tagging the archive is a big
> project.  I did not want to risk starting
> discussions with you if you might just go MIA
> six months later.

Of course, that's very sensible...


> surreal.  It is also too big for me to handle
> alone.  I am pleased and relieved to lace up my
> boots and join your army.

World domination!!


> However, we have a temporary but real problem.
> You started by defining tags to assign to
> packages.  I started by sorting packages into
> piles then naming the piles.  Your approach is
> smarter, but under my approach the work is
> nearly done.  And the two approaches are
> apparently not mixing.  In chemical terms, what
> we want is some detergent that makes debram
> soluble in debtags.  We want debram to mix.

I came to think that you can't define tags without tagging, and probably
also vice-versa: if you don't tag, you don't get insight on how to
organize tags; if you don't define some structure in your tag sets, it's
hard to remember which tags to apply, and you risk making groups that
only make sense to the tagger.  I'm really impressed by the work you've
done in this respect!


> Packages with similar purposes, similar
> audiences and/or similar dependencies tend to go
> together;

I've been browsing through debram.txt, and I saw that these terminal
groups really made sense.  A first idea I have for a merge is this:

 1) Given the list of all your terminal groups, see if they are all
    representable using intersections between different debtags facets.
    If not, then refine the facet structure to be able to represent your
    categories.
    Since you already mapped a huge part of the archive, working with
    your categories to test the facet structure is a good approximation
    of testing them with all the packages, except that it's *humanly
    possible*! :)  It could be very, very valuable!
 2) After upgrading the facet structure, make a single tag for any of
    your terminal groups, then load up the collection in tagcolledit and
    try to attach to the groups the proper faceted tags

This could be a way of porting the debram groups to debtags, having
debtags tags for almost any package.  Further facets not representable
in debram could be later added by people or developers navigating the
archive with Erich's packagebrowser, debtag-edit, some future extension
field of the debian/control file or whatever cool thing we may come out
designing,


> Unless someone has an alternate plan---and I do
> not deny that a good alternate plan exists; my
> ears are open---I see little else to do at the
> exact moment but to finish ramifying the debram
> as intended: complete through sarge.  It is hard
> to take concrete steps to merge debram into
> debtags until debram has a complete, checked
> body of tags to merge.  If this is wrong in your
> view, please tell me so; and if possible please
> suggest a workable alternative.

It sounds good to me, if you feel like you can do it.  Being an almost
suggestive division, it would be very hard for people to help, except
maybe that debtags-edit or Benjamin's packagebrowser could help in
getting insight on the packages you still haven't processed, in case
someone tagged them already.

In the next days I'd like to start working in point 1 of the above
proposal (if there won't be other ideas popping up in the list), which
would be a good critic exercise of the categories.  Then we'll see what
happens!



> QUESTION
>=20
> If this is correct, however, then here is my
> question: do we all now feel that the debtags
> stand on sufficiently firm ground, that it would
> be safe to abandon debram development after
> completion through sarge?  Continuing to develop
> two incompatible tagging systems in parallel
> seems a waste of scarce effort: apparently we
> all agree on this.  I want to develop debtags
> not debram, yet now I am spending all my Debian
> time on debram.  The question is, when and how
> can I stop?  After two years of steady
> development, the debram has some momentum; it is
> not so easy to stop.

So far, the debtags facets need consolidating and maturing a bit.  If
you run debtags-edit and start going through not-yet-tagged packages,
you'll see that some of them are difficult to tag using the current tag
set.  This is mostly because the current facet structure comes from the
tagging experience of Erich, myself and the few others that contributed
to the effort so far.  This includes being limited by the ranges of our
interests and our knowledge.

As our faceted orizons expand, I can see lots of new facets popping up
and opening the new problem of having too many facets to keep track of,
making interfaces insane again.  This is already starting to happen.

One idea would be to introduce a new level in the facet selection tree
to organize the facets themselves; another idea would be to first ask
the user what facets are relevant to him/her and then hide the others.

For now my idea is to just do as if it weren't a problem and then tackle
the problem when we can see it clearly.

I also keep caressing the idea of "outsourcing" to expert people the
handling of some facets: that would help the tagging job, and give a
very good work in various specific areas.  When showing debtags-edit to
Weasel, for example, he gave me lots of suggestions about security
related packages that I've never heard about.

Merging different tag sources AND vocabularies is instead already
implemented in the new debtags, and in Porto Alegre me and Pere made a
small simple script to convert Custom Debian Distributions metapackages
definitions into a debtags database.  That could be used to instantly
get debtags facets for all the various CDD distributions, and ship
debtags with a sources.list file listing all of them.

Another debtags problem is size: the whole suite has 2 libraries, 2
commandline tools and 2 GUI interfaces, and all of this is becoming
rather difficult to maintain and package, let alone to improve.  I'm
hoping that with time and further spreading of the system, hacking on
the code will become a community job and that subversion will see
commits by other people as well.  I'm also quite worried because I
haven't managed to obtain this with guessnet and the other simpler
packages I maintain, and I fear that it may be a problem on how I handle
my coding projects; however, I'm quite happy in seeing Benjamin
understand the code I write! :)

This is my perception of debtags grounds so far; of course Erich, Herv=E9
or others may have different views.


I'm happy to see that the projects could actually get merged, and sorry
for having written another long mail myself...


Ciao,

Enrico

--
GPG key: 1024D/797EBFAB 2000-12-05 Enrico Zini <enrico@debian.org>

--envbJBWh7q8WU6mo
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFA5e8a9LSwzHl+v6sRAnHIAJkByQ2KZquuHqqRm10IZfB/AICeowCdE7Qd
6kQ/iPC3DqZT7+6eonvZvGk=
=JXeW
-----END PGP SIGNATURE-----

--envbJBWh7q8WU6mo--