[buildd-tools-devel] Bug#620589: Bug#620589: Bug#620589: sbuild: option to turn off log filtering

[Roger Leigh]

tags 620589 + fixed-upstream pending
thanks

On Mon, Apr 04, 2011 at 12:41:28AM +0200, Jakub Wilk wrote:
> * Roger Leigh <rleigh at codelibre.net>, 2011-04-03, 01:26:
> >>Please add an option to turn off log filtering. Thanks in advance.
> >That's no problem.
> 
> Thanks.

Now done.  You can set $log_filter = 0; in sbuild.conf/.sbuildrc
to disable it.

> >>(To be honest, I don't understand why you decided to cripple
> >>logs with this "feature"...)
> >
> >Some people complained that build logs couldn't be diffed easily due
> >to the random part in the build directory name (which the filtering
> >makes constant).
> 
> Yes, I stumbled on this problem a few times, too. But:
> 
> 1. If you diff logs only ocasionally, you can easily fix the problem
> locally. Just use search&replace function in your $EDITOROFCHIOCE.
> Or write a <50 lines post-processing script in your
> $LANGUAGEOFCHOICE.
> 
> 2. If you diff logs on large scale, you need the current sbuild's
> log filtering is too naïve: there are other sources of diff noise,
> which sbuild cannot always fix without losing information (e.g.
> aptitude/apt-get/dpkg output). So one would up with writing a
> (sophisticated) log post-processor anyway.

I absolutely agree; total consistency between builds is probably
impossible to achieve, and we also put things like the date,
package versions etc. in there which will generally be changing.

There was a particular complaint about the variability and long
paths (the length may be more legitimate).  I implemented the
filtering as a solution to that.  Note that the filtering mechanism
is extensible to replacing any fixed pattern with a replacement,
so it can be extended to other strings as well if required.

> 3. sbuild _do_ lose information even now, because strings like
> CHROOT and BUILDDIR can occur naturally in a build log. See e.g.:
> https://buildd.debian.org/fetch.cgi?pkg=stunnel4&arch=ia64&ver=3:4.29-1&stamp=1260333411&file=log

This is a good point.  I know it's not totally foolproof, but I added
guillemots to the replacement text:

I: NOTICE: Log filtering will replace 'build/dadadodo-Zu8Yd5' with '«BUILDDIR»'
I: NOTICE: Log filtering will replace 'var/lib/schroot/mount/sid-amd64-sbuild-0f09fbd2-17fb-43c0-99e4-5b2cc5fbfa4d' with '«CHROOT»'
…
/usr/bin/make clean
make[1]: Entering directory `/«BUILDDIR»/dadadodo-1.04'
rm -f *.o a.out core dadadodo
make[1]: Leaving directory `/«BUILDDIR»/dadadodo-1.04'
 dpkg-source -b dadadodo-1.04

If you've got a better choice of replacement character, I can change it
--it's just not that widely used, and so means you can revert the
replacement a lot more reliably than without.

> 4. If want copy&paste filtered log snippets and send them to other
> people, I need to either undo the filtering (which cannot be done
> fully automatically, see 3.) or write an extra explanation about the
> transformation that were done. This is completely unnecessary
> burden.

True; hopefully turning it off with the above change will be OK for
you?

Note that the existing two replacements should not impact on the
utility of logs since they are build-specific paths which are not
reproducible.  The only time you would need to care about them is
if you are debugging a bug in sbuild itself if it's using the wrong
paths, or you need to examine a broken build, in which case you'll
have the needed information (location) right at the top of the log.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20110404/7bc5a5b1/attachment.pgp>