Let autopkgtests be gating for testing migration in Buster: heads-up and brain-dump

[Martin Pitt]

Hello Julien,

Julien Cristau [2017-01-03 22:23 +0100]:
> I'm not so sure about adding a feedback loop where
> britney says "this package would be a candidate, but first I'm going to
> request a CI test, then wait for some time for that result to be
> available, then make it an actual candidate" (at which point maybe the
> package set has changed so you're waiting for yet another test result).
> And if you're not talking about such a feedback loop

I am talking about such a feedback loop, yes. What's wrong with requesting and
waiting for another test result if the britney implementation of determining
which tests to run changes in the middle? That's exactly what you were
intending after all. (No other case comes to my mind where this could happen.)

> then it seems to me the CI system could figure out which tests to run on its
> own.  What am I missing?

We actually started with that in Ubuntu too (back then we used Jenkins and that
determined the tests), but it's just too annoying, brittle, and it's a very bad
design if two different entities try to do policy decisions -- it will happen
that e. g. the CI machinery starts to run a test while a package is (still)
uninstallable and thus generate unnecessary/bogus failures, and britney does
know about these. The other way around, britney might just eternally wait for a
result that the CI machinery never considered because someone set up a new
series on one side but not the other.

We've been through about four iterations of this, and the only thing that
actually made sense and works was to clearly separate policy and machinery.

Martin
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)