Bug#807270: mk-origtargz: create reproducible tarballs and --mtime option

[Hans-Christoph Steiner]

Osamu Aoki:
> Hi,
> 
> On Mon, Dec 07, 2015 at 08:07:53PM +0100, Hans-Christoph Steiner wrote:
>>
>>
>> Osamu Aoki:
>>> Hi,
>>>
>>> Second thought ...
>>>
>>> uscan/mk-origtargz/uupdate is not run during the binary package building
>>> process.  Does the reproducible build aims to create source package in
>>> reproducible way?
>>>
>>> If reproducible build is aiming for binary build reproducibility,
>>> changing behavior of uscan/mk-origtargz/uupdate has no impact.
>>
>> We want to have the whole process able to be inspected, including the
>> process of making the source tarballs.  But yes, binary reproducibility
>> is more important.  In this case, it is pretty easy to make reproducible
>> source tarballs, so I think its worth doing.
> 
> Please also remember that reproducing upstream content including the
> file time stamp is important factor.
> 
>>> On Mon, Dec 07, 2015 at 10:30:10PM +0900, Osamu Aoki wrote:
>>>
>>>> On Sun, Dec 06, 2015 at 10:21:04PM +0100, Hans-Christoph Steiner wrote:
>>> ...
>>>>> Whenever mk-origtargz is repacking a zipball, it should zero out the
>>>>> timestamps in the tar format so that the process produces the same
>>>>> tarball every time it runs.
>>>
>>> Why you need this?  unzip preserves file timestamps inside of zip
>>> archive.  Am I right?  Is this something we need to do for repacking of
>>> tar.gz?
>>
>> I believe unzip will preserve the timestamps.
> 
> So why you wish to overwrite mtime?  Does the upstream release zipball
> with different time stamp everytime user request to download?
> 
> Please be concrete on the needs with actual example package so we are
> not expanding on fantasy.

Yes, Google's http://googlesource.com website provides nice .tgz
download links for every commit, but those tarballs are different everytime.


>>>>> This can be done using tar's --mtime= flag.
>>>
>>> Yah, if it is needed.
>>>
>>>>> Additionally, it would be very useful if mk-origtargz also had a --mtime
>>>>> option which forced the tarball to be repacked using the date given to
>>>>> the --mtime="Wed Oct 28 10:12:27 2015 -0700" flag.  Here's an example of
>>>>> how to do that in perl:
>>>>>
>>>>> https://stackoverflow.com/a/16728218
>>>
>>> Well ... it is simpler than this as long as we know what date to set.
>>> Just run tar with --mtime option in the code with the reference file or
>>> date string.
>>
>> As long as mk-origtargz has an --mtime option, then we can use the most
>> appropriate date.  For example, with Android SDK packages, we can get
>> the git commit date of the release, since upstream does not post release
>> tarballs, only git tags.  It is this use case that made me want
>> mk-origtargz to support --mtime.
> 
> If we add features, we need to add infinite number of them unless there
> is a strong case which makes addition useful.  Does android SDK zip ball
> has rondom timestamp inside zipball?
> 
> Regards,
> 
> Osamu

http://googlesource.com uses the current date/time as the time stamp
each time you download it.  The timestamp is the mostly likely variation
when producing source tarballs from git/etc.

.hc