Bug#871806: uscan: (dpkg, git-buildpackage) accept/mangle/store signed git tags in cases where upstream does not publish detached sigs on tarballs

[Daniel Kahn Gillmor]

On Fri 2017-08-11 20:51:25 -0400, Osamu Aoki wrote:
>> In my experience, if the goal is to create a minimal "shallow clone
>> snapshot" of the git archive, it's quite often the case that i as the
>> packager already have a full clone of the upstream repo i'm working on.
>
> Yah but tool needs to be general and logic needs to be simple for
> maintenance.  uscan is already very complicated...

I think it's totally fair for uscan to implement a feature that only
works for a subset of users, as long as that subset is "people who do
things in a sensible and/or common way".

For instance, this is already the case for the upstream pgpsig feature,
which is excellent.  We're handling the case for upstreams who follow
that practices.

I'm proposing that we do something for debian developers with upstreams
who ship signed tags, when those DDs have a full clone of the upstream
git repo.  This is both sensible and common :)

>> I made the .gtsig of hddemux 0.3 with:
>> 
>>     git clone https://gitlab.com/dkg/hddemux
>
> I see full clone 1st ...

yes, that's correct.

>>     git clone --depth=1 --bare -b hddemux-0.3 file://$(pwd)/hddemux hddemux-0.3.git
>>     tar cz hddemux-0.3.git > hddemux-0.3.gtsig
>
> You just tar-up everything???? I thought you are going to check
> signature of tag against content...

sorry, i assumed that we understood that the git tag was already
correctly verified in the process of creating this .gtsig.

You can verify the tag with either "git tag -v" or "git verify-tag",
yes?

Anyway, it's in the verification stage that we really care about
validating the signature on the tag.  And i did include the tag
verification ("git tag -v") there:

>>      tar xz < hddemux-0.3.gtsig 
>>      ( cd hddemux-0.3.git && \
>>        git tag -v hddemux-0.3 && \
>>        git archive --format=tar --prefix=hddemux-0.3/ hddemux-0.3 | sha256sum )
>>      rm -rf hddemux-0.3.git/
>
> Does hush given tar exactly the same as one signed in tag ... I guess I
> need to RTFM.

do you mean s/hush/hash/ ?  i don't think i understand the question.

>> and testing whether (a) the tag verified, and (b) the the output of
>> sha256sum matches against:
>> 
>>      zcat < hddemux_0.3.orig.tar.gz | sha256sum -
>> 
>> (above, i'm testing the digest of the uncompressed tarball.  testing the
>> compressed version would also be pretty easy)
>
> Compression depends on compression parameter which may be different for
> different arch.  Are you sure you tested your thought works???

yes, the digest of the uncompressed tarball matched the digest of the
output of "git archive".  I did test that with hddemux 0.3 at least :)

> Can you point me to any resource which indicate your way of verifying
> tag via tar is the right way...  Please educate me on this.

i'm not verifying the tag via tar.

i'm verifying the tag, and then checking that the tarball generated from
the tag matches the orig.tar .  This is (intended to be) equivalent to
verifying a cryptographic signature over a tarball, since (almost) the
same data is being signed.

>> Many upstream maintainers can be convinced to make the HEAD of their
>> repo DFSG-free -- they understand that some weird jpg some previous
>> developer committed a few years ago isn't really kosher.
>
> Many ... yes,  All ... no.
>
> uscan is a tool.  Why limit its capability?

Because we need something functional for a common use case?  All good
tools are limited, that's what makes them good tools.  they're good at
doing a particular thing.  I don't understand why you're raising this
question here -- i'm pretty sure you don't want debian to ship
non-dfsg-free data in the archive, right?

>> but that doesn't mean that their history is DFSG-free.  My concern was
>> about avoiding shipping non-DFSG-free history, when the tag we want to
>> ship is itself DFSG-free.
>
> Why send entire history?

That's why i'm proposing basing this on a shallow clone -- so that we
don't need to send the entire history.  But to verify a git tag, you
need at least to know the shallow clone, or else you can't verify the
tag.

> I feel strange .... At least creating such big file to veriy its
> current content beat the purpose of making cryptographic signature.

I agree -- the .gtsig is actually the equivalent to [orig.tar.gz +
orig.tar.gz.asc] (plus it contains a few dangling pointers to the git
history).  As constructed above, it's *not* the same thing as the
orig.tar.gz.asc itself.

> When I heard you talking signature, I thought you are talking to create
> a file with a signature on a tagged contents.  But here you are creating
> something different.  To be honest, my knowledge is weak on this.  But I
> am starting to feel you nor me have concrete idea what needs to be
> done.

Maybe a voice/video chat would help to explain this.  I think both of us
have at least half the picture.  perhaps we could put our ideas together
and come up with something that solves some problems?

Feel free to mail me and suggest a time that would work for you to chat!

    --dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/devscripts-devel/attachments/20170814/51a4a378/attachment.sig>