Let autopkgtests be gating for testing migration in Buster: heads-up and brain-dump

[Martin Pitt]

Hello Antonio,

Antonio Terceiro [2016-12-21 18:13 -0200]:
> > 0) figure out how to test all of this without breaking the real
> > instances (hints more than welcome).
> > 1) fix autopkgtest to enable --apt-suite (next to the current --apt-pocket)
> > In parallel:
> > 2a) getting a testing suite up for debci and extend debci to be aware of
> > the additional arguments for autopkgtest
> > 2b) let britney generate a list of tests it would like to perform
> > 2c) align on the transfer mechanism between britney(1) and debci
> > 3) enable debci to swallow the commands from britney
> > 4) enable the policy in britney
> 
> Sure. My main concern here is knowing exactly what the interface between
> britney and debci is going to be. Obviously we don't want a circular
> dependency, so it seems that you are going towards britney knowing how
> to deal with debci, and not the other way around.

Correct. debci should provide the machinery to run tests, i. e. accept test
requests via AMQP (as it does today, except that britney would send the
requests instead of debci's cron), run the test, and export the results
(https://ci.debian.net/data/ works fine, britney can read that and it's fairly
close to reading swift like the Ubuntu implementation does). debci should not
interpret the results and do policy decisions, that's britney's domain and thus
debci should not know about britney.

> On the debci side, we would need:
> 
> - when testing to see if package X can be let into testing, britney
>   needs a way to say a) "test package X from unstable on a testing base"
>   and b) "test package Y from testing with X from unstable".

That's provided with the "triggers" option, Paul already explained that. The
trigger contains the "reason(s)" why a test was started, which could either be
a newer version of the tested package itself, or a change of any of its
dependencies.

>   there would be one Y for each of the reverse dependencies of X, and
>   that list would be generated by britney, I assume.

Right. britney has that information anyway for installability testing, it's
quite straightforward to generate:

  https://git.launchpad.net/~ubuntu-release/britney/+git/britney2-ubuntu/tree/britney2/policies/autopkgtest.py#n348

> - a way for britney to "inject" these test requests, and a way for it to
>   get their results back. This would probably require having some sort
>   of identifier generated by britney that can be used by later to match
>   the request to the results.

Right. The AMQP request contains the triggers, debci translates them as
autopkgtest --env arguments (--env=ADT_TEST_TRIGGERS=foo/1.0-2 bar/2.0-3):

  https://git.launchpad.net/~ubuntu-release/+git/autopkgtest-cloud/tree/worker/worker#n327

and these envs ends up in results.tar which britney reads, and with that it can
map a result back to a run for a particular trigger.

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20161227/6cbd44a8/attachment.sig>