Bug#844459: autopkgtest: Please add autopkgtest-virt-uchroot

[Johannes Schauer]

Hi,

Quoting Martin Pitt (2016-12-08 17:43:39)
> Maybe I misunderstood you above, but you said that sbuild was creating the
> "session", which I assumed was the schroot session that it uses to build the
> package and run its autopkgtest in.
> 
> If sbuild *does* control the schroot session, then it's not autopkgtest's
> business to control (revert/stop) it.
> 
> If sbuild does *not* control the schroot session, then I'm afraid I
> don't understand the problem -- autopkgtest should receive the ^C,
> clean up, then exit, and sbuild either sees that it exited and can then do its
> cleanup or it receives the SIGINT by itself and does its cleanup in parallel.

ah okay, just a misunderstanding then. Let me also elaborate from my side.

So what the sbuild autopkgtest backend currently does, is to run the respective
autopkgtest-virt-* executable and then communicate with it via stdin/stdout.
Sbuild does *not* do any schroot-specific stuff there as that is all abstracted
away by autopkgtest. In fact, sbuild doesn't even do anything different
independent of the backend used. That it might've been told to use the schroot
autopkgtest backend is stored in a variable somewhere but sbuild doesn't do
anything special because it knows that value. In fact it should act the same
way independent of the backend. The only differences of course should come from
the capabilities announced by autopkgtest.

So indeed sbuild does not create the schroot session created by autopkgtest
other than through autopkgtest itself. So the problem here is twofold:

Firstly, because the autopkgtest-virt-* program is in the same process group it
will also receive the Ctrl+C sent by the user and will immediately clean up and
exit. The problem with autopkgtest doing that from the sbuild point of view is,
that sbuild doesn't expect that the underlying chroot vanishes without sbuild
telling it to do so. Now you could say that this just means that sbuild has to
be adapted to be able to handle the backend behaving that way. Unfortunately
that would mean for the autopkgtest backend to be more limited than other
backends. For example there exists an sbuild hook that is triggered when the
build fails.  An sbuild user might want to run this hook to automatically
collect statistics of the build environment after the build was aborted. This
doesn't seem to be possible right now with the autopkgtest backend because at
the point that the hook is to be executed, the backend already terminated
itself. It would be great if sbuild could tell the autopkgtest container to
please ignore any SIGINT (or any other signals outside signals that would lead
to autopkgtest shutting down the container). I could imagine this to be
implemented as a command line option or an environment variable.  Would you be
open to patches?

Secondly, there is a uchroot specific problem here. For the uchroot backend,
cleaning up or reverting the session means to recursively remove the unpacked
rootfs. This will fail if there are still applications running at the time that
hook_cleanup() is called which keep some filehandles open and thus prevent
clean removal. Conversely, there exists the problem that applications might
still be started while the cleanup is in process. These will then likely fail
halfway through because the stuff from the filesystem they need is not
available anymore. Ideally, I'd like to be able to check in the hook_cleanup()
function whether anybody is still having any open filehandles on the chroot and
wait until every program finished executing. And I would also like to not
attempt starting any programs inside the chroot while hook_cleanup() is in the
process of doing the cleanup. Which kind of communication can I use between
these two functions such that they are able to determine whether the other is
currently being executed or not?

I hope things are clearer now. :)

Thanks!

cheers, josch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20161208/7a0dc04a/attachment-0001.sig>