[Reproducible-commits] [blog] 01/01: First full draft of week 5 reprotest report

Wed Jun 29 04:00:45 UTC 2016

This is an automated email from the git hooks/post-receive script.

ceridwen-guest pushed a commit to branch master
in repository blog.

commit 759884c263cf16023f2dd70f0a37b1e7984a5b17
Author: Ceridwen <ceridwenv at gmail.com>
Date:   Wed Jun 29 00:00:37 2016 -0400

    First full draft of week 5 reprotest report
---
 drafts/people/ceridwen/reprotest_week5.mdwn | 122 ++++++++++++++++++++++++----
 1 file changed, 107 insertions(+), 15 deletions(-)

diff --git a/drafts/people/ceridwen/reprotest_week5.mdwn b/drafts/people/ceridwen/reprotest_week5.mdwn
index f48eec5..7977300 100644
--- a/drafts/people/ceridwen/reprotest_week5.mdwn
+++ b/drafts/people/ceridwen/reprotest_week5.mdwn
@@ -1,11 +1,76 @@
-[[!meta title="Reorganizing the autopkgtest code"]]
-[[!meta date="Mon, 20 Jun 2016 14:30:02 -0400"]]
+[[!meta title="First steps towards getting containers working"]]
+[[!meta date="Tue, 28 Jun 2016 19:19:59 -0400"]]
 [[!tag reproducible_builds Debian reprotest Outreachy]]
 Author: ceridwen
 
-If the authors of autopkgtest had *deliberately* chosen a structure
-designed not to work with setuptools, I don't know if they could have
-done a better job.  As I discussed [last
+The 0.1 alpha release of reprotest has been accepted into Debian
+unstable, so it should be available for install now.
+
+I've been working on redesigning reprotest so that it runs commands
+through autopkgtest's adt_testbed interface.  For the most part, I
+needed to replace explicit calls to Python standard library functions
+for copying files and directories with calls to
+`adt_testbed.Testbed.command()` with `copyup` and `copydown`, and to
+use `Testbed.execute()` and `Testbed.check_exec()` to run commands
+instead of `subprocess`.
+
+To test reprotest on the actual containers requires having containers
+constructed for this purpose.  autopkgtest has a test that builds a
+minimal chroot.  I considered doing something like this approach or
+using BusyBox.  However, I have a Python script that mocks a build
+process, which requires having Python available in the container, and
+while I looked into
+[busybox-python](https://github.com/odise/busybox-python) and
+[MicroPython](https://github.com/micropython/micropython) to keep the
+footprint small, I decided that for now this would take too much work
+and decided to go straight to the autopkgtest recommendations for
+building containers, [mk-sbuild](https://wiki.debian.org/mk-sbuild)
+and [vmdebootstrap](https://wiki.debian.org/vmdebootstrap).  This
+means that to get the tests run requires some manual setup at the
+moment.  In the long run, I'd like to improve that, but it's not an
+immediate priority.  While working on adding tests for the other
+containers supported by autopkgtest, I also converted to
+[py.test](https://pytest.org/latest/index.html) so that I could use
+[fixtures](https://pytest.org/latest/fixture.html) and
+[parametrization](https://pytest.org/latest/parametrize.html) to run
+the Cartesian product of each variation with each container.
+
+With tests written, I started trying to verify that my new code
+worked.  One problem I encountered while trying to debug was that I
+wasn't getting full error output.  In `VirtSubproc.check_exec()`,
+`execute_timeout()` acts something like a `Popen()` call:
+
+    (status, out, err) = execute_timeout(None, timeout, real_argv,
+                                         stdout=stdout, stderr=subprocess.PIPE)
+
+    if status:
+        bomb("%s%s failed (exit status %d)" %
+             ((downp and "(down) " or ""), argv, status))
+    if err:
+        bomb("%s unexpectedly produced stderr output `%s'" %
+             (argv, err))
+
+The problem with this is that if the call returns a non-zero exit
+code, which is typical for program failures, stderr doesn't get
+included in the error message.
+
+I changed the first if-block to:
+
+    if status:
+        bomb("%s%s failed (exit status %d)\n%s" %
+             ((downp and "(down) " or ""), argv, status, err))
+
+Another example is that autopkgtest calls `schroot` with the `--quiet`
+flag, which in one case was making `schroot` fail without any output
+due to a misconfiguration.  I'm still trying to find and eliminate
+more places where errors are silenced.
+
+autopkgtest was designed to be installed with Debian's packaging
+system, which handles arbitrary files and directory layouts.
+Unfortunately, setuptools is *completely* different in a way that
+doesn't work well with autopkgtest's design.  (I'm sure this is partly
+because setuptools has to support all the different major OSes that
+run Python, including Windows.)  As I discussed [last
 week](https://reproducible.alioth.debian.org/blog/posts/people/ceridwen/reprotest_week4/),
 autopkgtest has Python scripts in `virt/` that are executed by
 `subprocess` calls in `adt_testbed`.  Because these scripts import
@@ -13,16 +78,16 @@ files from `lib/`, there needs to be an `__init__.py` in `virt/` to
 make it into a package and a `sys.path` hack in each script to allow
 it to find modules in `lib/`.  Unfortunately, setuptools *will not
 install* this structure.  First, setuptools will not install any file
-without a `.py` extension into a package.  Theoretically, thisis
-fixable, they are Python scripts so I could rename them.
-(Theoretically, there's supposed to be some workaround involving
-`MANIFEST.in` or `package_data` in `setup.py`, but I have yet to find
-any documentation or explanation giving a method for installing
-non-Python files inside a Python package.)  Second, however,
-setuptools does not preserve the executable bit when installing
-package files.  The obvious workaround, changing the `subprocess`
-calls so that they invoke `python virt/foo.py` rather than
-`virt/foo.py` requires changing all the internal calls in the
+without a `.py` extension into a package.  Theoretically, this is
+fixable, the files in `virt/` are Python scripts so I could rename
+them.  (Theoretically, there's supposed to be some workaround
+involving `MANIFEST.in` or `package_data` in `setup.py`, but I have
+yet to find any documentation or explanation giving a method for
+installing non-Python files inside a Python package.)  Second,
+however, setuptools does not preserve the executable bit when
+installing package files.  The obvious workaround, changing the
+`subprocess` calls so that they invoke `python virt/foo.py` rather
+than `virt/foo.py` requires changing all the internal calls in the
 autopkgtest code, which I'm loathe to do for fear of breaking it.
 (It's not clear to me I can easily find all of the calls, for
 starters.)
@@ -46,3 +111,30 @@ of all the autopkgtest code to import the code in the scripts rather
 than running it through `subprocess` calls.  I'm reluctant to do this
 because I think it's almost certain to break things that will require
 significant work to fix.
+
+Getting setuptools to install the autopkgtest code correctly is one
+blocker for the next release.  Another is that autopkgtest's handling
+of errors during the build process involves closing the
+`adt_testbed.Testbed` so it won't take further commands.
+Unfortunately, this handling runs *before* any cleanup code I write to
+run outside it, which means that at the moment errors during the build
+will result in things like disorderfs being left mounted.
+
+The last release blocker is that `adt_testbed` doesn't have any way to
+set a working directory when running commands.  For instance, the
+`virt/schroot` script always calls `schroot` with `--directory=/`.  I
+thought about trying to use absolute paths, but decided this was
+unintuitive and impractical.  For the user, this would mean that
+instead of running something simple like `make` in the correct
+directory, they would have to run `make
+--file=/absolute/path/to/Makefile` or something similar, making all
+paths absolute.  I worry that some build scripts wouldn't handle this
+correctly, either: for instance, running `python setup.py` from a
+different directory can have different effects because Python's path
+is initialized to contain the current directory.  Changing this is
+going to require going deeper into the autopkgtest code than I'd
+hoped.
+
+I intend to try to resolve these three issues over the next week and
+then prepare the next release, though how much progress I make depends
+on how thorny they turn out to be.

-- 
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/reproducible/blog.git

