Bug#852820: Testsuite-Restrictions field is hard to use
ijackson at chiark.greenend.org.uk
Tue Jan 31 13:59:47 UTC 2017
Iain Lane writes ("Re: Bug#852820: Testsuite-Restrictions field is hard to use"):
> I don't, or not really - see below. I plan on using this field
> externally to choose between a couple of available backends to dispatch
> to when constructing autopkgtest invocations
Yes, I understand that. But I think this field is too tailored to
your specific use case and too ripe for misuse.
> > https://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/plain/doc/README.package-tests.rst says:
> > Tests which declare unknown restrictions will be skipped. See below
> > for the defined restrictions.
> So it would seem to me (unless I'm missing something in there like an
> exception for x-prefixed items that I can't see ATM) that this would be
> legal and as a result I was not really expecting arbitrary values to be
> present. Again, I'm not going to be doing that, so it's not a problem
> now and this is apparently not currently implemented in autopkgtest
> itself either, since dgit's tests are being run.
Only a few of dgit's tests have these restrictions. The tests with
those restrictions are not run by Debian's CI. They can be run in
ad-hoc situations of various kinds.
Declaring test restrictions, using a private bit of the namespace, in
this way, seems like an appropriate way to achieve that effect.
For the avoidance of doubt: I'm not saying that your program would
mishandle correct packages. (Is it happy with dgit's test suite,
despite the presence of some un-runnable tests?)
But as I say this field seems to invite people to use it for deciding
whether the package is testable, but it is not useable for that
To put it another way: why not publish the intersection of the
restrictions ? Why not also publish the union or intersection of the
features ? As I wrote:
> > > If we do need more information in Sources.gz then maybe the set of
> > > combinations of restrictions ought to be listed. But then the
> > > features might be useful too.
ISTM that your real problem is that you don't have an efficient way to
access debian/tests/control. Perhaps the right answer is to provide
an additional side channel for this information, rather than burdening
the Sources file. Many many, non-testing-related, programs need to
read Sources. I think the test metadata is rather too rich to express
> > But I certainly agree that this should have probably been discussed
> > more widely, which is something I overlooked, sorry about that. And
> > agree completely with your two points above. So I'll be adding an
> > entry to the FAQ detailing the process to add new information to
> > the .dsc file (and probably to the .changes and other interchange
> > formats).
> Yes, I concede that I should have posted on the list. I discussed with
> Martin first, but that is not enough. Sorry about that.
More information about the autopkgtest-devel