[buildinfo-spec] 01/01: deb-src-hash: alternative proposal

[Ximin Luo]

This is an automated email from the git hooks/post-receive script.

infinity0 pushed a commit to branch master
in repository buildinfo-spec.

commit e08d6cbe3954f3e57a670a02a66e2c8cdeab63a6
Author: Ximin Luo <infinity0 at debian.org>
Date:   Mon May 8 18:11:07 2017 +0200

    deb-src-hash: alternative proposal
---
 notes/deb-src-hash.rst | 80 ++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 65 insertions(+), 15 deletions(-)

diff --git a/notes/deb-src-hash.rst b/notes/deb-src-hash.rst
index 112cb21..289956d 100644
--- a/notes/deb-src-hash.rst
+++ b/notes/deb-src-hash.rst
@@ -26,8 +26,8 @@ Therefore, when building, we must add this information to
 
 How to achieve this?
 
-Proposal
-========
+Proposal A
+==========
 
 1. ``dpkg-source --before-build`` should write ``debian/source/sha256`` before
    applying patches, being the sha256 of the unsigned .dsc, calculated from the
@@ -54,16 +54,66 @@ Proposal
    ``debian/rules build`` directly e.g. for debugging, but gives a very strong
    indicator to Do Things Properly.
 
-We must also fix ``dpkg/devscripts`` tools, and any packaging rules, to ensure
-that they do not touch Debian packaging files during the build. Otherwise this
-would disrupt the source hash for future (source or binary) builds. For example
-``dpkg-buildpackage -S`` (as of 1.18.3) touches ``debian/`` for some reason.
-
-It would be good to check the above property after a build, but I can't figure
-out a way good way to do this. Calculating the hash requires a clean tree,
-which is not generally the case post-build. One ugly way would be to generate
-``debian/source/files`` including the same metadata that ``tar`` would store,
-and then compare this just before running ``dpkg-deb`` or something. Perhaps
-it's best to just omit this. At least, (1.a) allows us to catch it at the start
-of the next build, and tools like sbuild/pbuilder always build from a tree
-directly unpacked from the .dsc anyways.
+Proposal B
+==========
+
+Proposal A is problematic because step (1) can be very expensive and we might
+not want to do this every time we run ``dpkg-buildpackage -b``. Here is a
+cheaper and simpler alternative:
+
+1. ``dpkg-buildpackage`` before calling ``dpkg-source --before-build`` should
+   check to see if there is a corresponding ``.dsc`` in the parent directory,
+   with the same filename that ``dpkg-source -b .`` would produce.
+
+   If it does not exist, then emit a warning: "no source package, your .debs
+   won't have the Source-Hash field".
+
+2. If it does exist, then check that it contains the same filelist (including
+   metadata like sizes and timestamps, which are cheap-to-access from the
+   archive) as what a call to ``dpkg-source -b .`` now *would* archive - but
+   don't do the actual archiving operation, which can be expensive.
+
+   (Unfortunately, this may not work well for version 1.0 packages, since
+   calculating the filelist plus metadata requires extracting the ``diff``
+   tarball and applying it to the filelist of the ``orig`` tarball. However it
+   should be possible to do this only using the metadata of the decompressed
+   patch, without actually applying the patch on-disk.)
+
+   If the filelists are different, then emit a warning: "existing source
+   package differs from the tree being built".
+
+3. If no warnings were emitted, we can communicate to ``dpkg-gencontrol``
+   (perhaps by an environment variable) that it can hash the relevant ``.dsc``
+   and add this to ``DEBIAN/control``.
+
+   Furthermore, if this information isn't available then ``dpkg-gencontrol``
+   can output a warning, in addition to the other warnings above.
+
+4. As per (4) in Proposal A, lintian should emit an ``ERROR`` for binary
+   packages that do not contain these hashes.
+
+Post-build
+==========
+
+With both these proposals, in order for a second build to work correctly,
+``debian/rules clean`` must work "perfectly", to restore the original tree
+after a build. Sometimes people are a bit slack in making this work, it is not
+a policy requirement, and some packaging tools like ``dh_autoreconf_clean``
+completely contradict this requirement. [1] However if it doesn't work
+properly, both proposals will detect this scenario, prompting the developer to
+re-build the ``.dsc``, or re-extract it over the current directory to restore
+the original timestamps.
+
+Also, ``--after-build`` must completely undo the effects of ``--before-build``.
+This is the case for Debian packages, which reset timestamps when applying
+patches, so that these don't need to be explicitly reverted when unapplying
+them. If the developer manually uses quilt, this *will* change the timestamps,
+but this would also be detected by both proposals and a warning emitted.
+
+Finally, tools like sbuild/pbuilder never try to do a second build after the
+first one anyway, so this scenario will rarely be a problem in practise.
+
+[1] At least, ``dpkg-buildpackage`` supports this requirement, and does not
+itself irreversibly touch source files. (At a previous version 1.18.3, ``-S``
+touched ``debian/`` for some reason, but this appears now to be fixed in the
+latest versions.)

-- 
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/reproducible/buildinfo-spec.git