Bug#795274: autopkgtest: please simplify the UI for the simplest use case

[Martin Pitt]

Hello Antonio,

Antonio Terceiro [2015-08-12 11:15 -0300]:
> In order to ease getting new people into writing tests, I think we
> should improve the UI. A few problems I have with it as of currently:
> 
> - the command to run tests does not match the package name: you install
>   `autopkgtest`, but then needs to call `adt-run`.

Indeed, this has been a long-standing wart. Originally the project was
called autodebtest, and adt-run and $ADT_* still stem from that. But
we can't possibly rename everything to apt-* and APT_* as that would
be utterly confusing.

Simplifying the UI was also requested in e. g.
https://launchpad.net/bugs/1453509 and I greatly sympathize. At some
point we just need a more robust and non-backwards compatible CLI.

> - renaming adt-run to `autopkgtest` (but of course keeping `adt-run` as a
>   synlink to `autopkgtest` to keep backwards compatibility)

+1. I'd reserve this "new" name for the incompatible new and
simplified UI. This would:

  - Only accept at most one action argument (defaulting to .)
  - Do away with the mandatory built/unbuilt-tree difference, and
    autodetect that instead
  - Accept auxiliary arguments like debs, --setup-commands etc. in
    arbitrary order (LP #1453509)
  - Possibly drop the non-standard --- separator and use -- instead.

The CLI is already completely separated in lib/adt_run_args.py, so the
new CLI could be implemented in lib/autopkgtest_args.py and build the
same options dictionary. Over time we can then phase out adt-run,
delete lib/adt_run_args.py, and get rid of quite a lot of
runner/adt-run code which exists to handle/track multiple action args.

> - make "./" the default input (while making sure that the current
>   directory is a valid source package

This seems fine to me. But it cannot work with the current
multi-action CLI.

> - make "null" the default backend.

This has a big nack from me, I'm afraid. The null backend is just
about the worst possible thing: It does not work for the vast majority
of packages (as almost all of them have test dependencies) when run as
the user, it is prone to destroy your machine if you run it as root
(which you should never EVAR do), and it produces unreliable results
as there is no defined test environment. It is really only mostly
there for testing autopkgtest itself, or if you have all test deps
with correct versions already installed and know what you are doing.

However, I would be fine with approaching this from another angle: We
currently have the possibility to "source" arbitrary parts of the
command line from a file, with @. E. g.

  adt-run foo.dsc @adt/lxc_runner.conf

which contains e. g. "--- lxc -s adt-sid" (each option on a new line,
but we could support spaces too). I'd be fine with reading e. g.
~/.config/autopkgtest.conf and getting a default runner with configs
from there. So you only have to set this up once.

WDYT?

Thanks,

Martin
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20150820/4a51bcbd/attachment-0001.sig>