Bug#871806: uscan: (dpkg, git-buildpackage) accept/mangle/store signed git tags in cases where upstream does not publish detached sigs on tarballs

[Daniel Kahn Gillmor]

Package: devscripts
Priority: wishlist
Control: affects -1 + dpkg git-buildpackage pristine-tar
X-Debbugs-Cc: dpkg at packages.debian.org, git-buildpackage at packages.debian.org, pristine-tar at packages.debian.org

Hi devscripts, dpkg, git-buildpackage, pristine-tar folks--

It's awesome to see the progress made on tracking upstream cryptographic
signatures via uscan and dpkg in debian.  This provides a dataset for
cryptographic provenance that can be useful for auditing.

We're handling detached OpenPGP signatures for tarballs at the moment,
but not all upstreams provide this particular form of cryptographic
assurance.

Some upstreams do provide cryptographic signatures, but only in git
tags.

I'm not sure exactly how to do this, but what i'd like to see is a way
for us to record and make use of signed git tags in the same way.

I'm opening this bug report in the hopes of starting discussion about
how to best do it.

The text below assumes the following:

 * upstream maintains their sources in a git repository.

 * upstream either doesn't produce tarballs at all, or they produce
   tarballs from their git repository using something like "git
   archive".  For the sake of the argument here, let's assume that they
   don't produce tarballs at all.

 * upstream does indicate releases in the form of OpenPGP-signed git
   tags.

 * The releases offered by upstream correspond to the "upstream
   versions" that are uploaded into debian.



Here's an extremely rough and inefficient approach (which i haven't
implemented, as this is in brainstorming phase).  I've probably even got
some of the terminology wrong, or the dataflows backward:

 * we document how we generate a debian "upstream tarball" from a git
   tag.  for example, we put this in debian/upstream/vcs-gen-tarball:

    git archive --format=tar --prefix=${projname}-${version} ${tagname} | gzip -9n

 * make a shallow clone of the git archive at the tag, including the
   tag. (i've confirmed that a signed git tag in a shallow repo does
   validate correctly).

     git clone --bare --depth 1 -b ${tagname} \
            file://path/to/upstream/${projname}.git ${projname}-${version}.git


 * create an archive of the shallow clone, combined with the command to
   generate the tarball (we can call this a "gtsig")

     rm -rf ${projname}-${version}.git/hooks
     cp debian/upstream/vcs-gen-tarball ./${projname}-${version}.git
     tar cz ./${projname}-${version}.git > ./${projname}-${version}.gtsig

 * write a simple tool to verify an orig.tar.gz against a signing key
   and a gtsig, by extracting the "shallow clone" of the git repository,
   verifying git tag -v, using git-archive, and then comparing the
   results.


Some of the outstanding concerns:

 * what if there is non-DFSG-free data in the upstream git repo?  We
   want to make sure we avoid shipping it to our mirrors.  that's why i
   was leaning toward the "shallow clone", but if there are other
   techniques, i'd be curious to hear them.

 * the .gtsig will be quite large -- roughly the same size as the
   orig.tar.gz.  Is it possible to make it smaller by just storing the
   "delta" needed to recreate the shallow clone from the orig.tar.gz?
   Or is it possible (though dirty) to ship the .gtsig itself as the
   orig.tar.gz?  that smells like trouble, because you couldn't
   reconstruct the sources without having git available.

 * the .gtsig itself will show "verified" but it could contain some data
   that isn't actually covered by the tag.  Upon verification, how do we
   make sure it's clean?  (i note that OpenPGP signature files also have
   covert channels where they can carry unsigned material, so this might
   not be introducing a new bug in general).

 * is "git archive" guaranteed to produce deterministic output?

What do folks think?  I'm sure i'm not the first person to think about
this, but i don't know whether there is any existing work done on it
either.  Pointers, thoughts, discussion welcome.

        --dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/devscripts-devel/attachments/20170811/fabc2bd4/attachment.sig>