Bug#807270: mk-origtargz: create reproducible tarballs and --mtime option

[Osamu Aoki]

Hi,

On Thu, Dec 10, 2015 at 11:42:46AM +0100, Hans-Christoph Steiner wrote:
> 
> 
> Osamu Aoki:
> > Hi,
> > 
> > On Mon, Dec 07, 2015 at 08:07:53PM +0100, Hans-Christoph Steiner wrote:
> >>
> >>
> >> Osamu Aoki:
> >>> Hi,
> >>>
> >>> Second thought ...
> >>>
> >>> uscan/mk-origtargz/uupdate is not run during the binary package building
> >>> process.  Does the reproducible build aims to create source package in
> >>> reproducible way?
> >>>
> >>> If reproducible build is aiming for binary build reproducibility,
> >>> changing behavior of uscan/mk-origtargz/uupdate has no impact.
> >>
> >> We want to have the whole process able to be inspected, including the
> >> process of making the source tarballs.  But yes, binary reproducibility
> >> is more important.  In this case, it is pretty easy to make reproducible
> >> source tarballs, so I think its worth doing.
> > 
> > Please also remember that reproducing upstream content including the
> > file time stamp is important factor.
> > 
> >>> On Mon, Dec 07, 2015 at 10:30:10PM +0900, Osamu Aoki wrote:
> >>>
> >>>> On Sun, Dec 06, 2015 at 10:21:04PM +0100, Hans-Christoph Steiner wrote:
> >>> ...
> >>>>> Whenever mk-origtargz is repacking a zipball, it should zero out the
> >>>>> timestamps in the tar format so that the process produces the same
> >>>>> tarball every time it runs.
> >>>
> >>> Why you need this?  unzip preserves file timestamps inside of zip
> >>> archive.  Am I right?  Is this something we need to do for repacking of
> >>> tar.gz?
> >>
> >> I believe unzip will preserve the timestamps.
> > 
> > So why you wish to overwrite mtime?  Does the upstream release zipball
> > with different time stamp everytime user request to download?
> > 
> > Please be concrete on the needs with actual example package so we are
> > not expanding on fantasy.
> 
> Yes, Google's http://googlesource.com website provides nice .tgz
> download links for every commit, but those tarballs are different everytime.

OK.  This is deprecated source but now you are tyalking about not just
zapball but tarball.

> >>>>> This can be done using tar's --mtime= flag.
> >>>
> >>> Yah, if it is needed.
> >>>
> >>>>> Additionally, it would be very useful if mk-origtargz also had a --mtime
> >>>>> option which forced the tarball to be repacked using the date given to
> >>>>> the --mtime="Wed Oct 28 10:12:27 2015 -0700" flag.  Here's an example of
> >>>>> how to do that in perl:
> >>>>>
> >>>>> https://stackoverflow.com/a/16728218
> >>>
> >>> Well ... it is simpler than this as long as we know what date to set.
> >>> Just run tar with --mtime option in the code with the reference file or
> >>> date string.
> >>
> >> As long as mk-origtargz has an --mtime option, then we can use the most
> >> appropriate date.  For example, with Android SDK packages, we can get
> >> the git commit date of the release, since upstream does not post release
> >> tarballs, only git tags.  It is this use case that made me want
> >> mk-origtargz to support --mtime.
> > 
> > If we add features, we need to add infinite number of them unless there
> > is a strong case which makes addition useful.  Does android SDK zip ball
> > has rondom timestamp inside zipball?
> > 
> > Regards,
> > 
> > Osamu
> 
> http://googlesource.com uses the current date/time as the time stamp
> each time you download it.  The timestamp is the mostly likely variation
> when producing source tarballs from git/etc.

OK so your feature request is to have such option not just for zip but
for all archive.  That makes some sense to me now.

Good night.  I need some time to think.

Osamu