[Reproducible-commits] [misc] 01/01: reports: Update documentation and tools
Jérémy Bobbio
lunar at moszumanska.debian.org
Sun Sep 13 07:22:01 UTC 2015
This is an automated email from the git hooks/post-receive script.
lunar pushed a commit to branch master
in repository misc.
commit 878f2d06650bfade159e174ccc938de9a4d4f89d
Author: Jérémy Bobbio <lunar at debian.org>
Date: Sun Sep 13 09:18:06 2015 +0200
reports: Update documentation and tools
---
reports/README | 21 +++++++++++++++++++++
reports/bin/newly-reproducible | 5 +++--
reports/bin/review-bugs | 15 ++++++++++-----
reports/template.mdwn | 14 ++++++++++----
4 files changed, 44 insertions(+), 11 deletions(-)
diff --git a/reports/README b/reports/README
index 09a5552..9832a29 100644
--- a/reports/README
+++ b/reports/README
@@ -21,6 +21,7 @@ Process
There's a script that will use the history in the
reproducible.debian.net database to figure this out. Usage:
+ $ rm -f reproducible.db # remove previously downloaded database
$ wget https://reproducible.debian.net/reproducible.db
$ bin/newly-reproducible
@@ -30,6 +31,14 @@ Process
might give some false positive: packages might have been
reproducible before, broken by a toolchain upload later fixed.
+ To download all the changelogs, one can do:
+
+ $ mkdir changelogs
+ $ for url in $(bin/newly-reproducible | sed -n -e 's,.*<,,;s,>,,p'); do
+ p=${url%/*}; p=${p##*/};
+ torsocks wget -O changelogs/$p $url;
+ done
+
One can look at the history of the tests by typing:
$ bin/history ghc # defaults to unstable
@@ -43,6 +52,7 @@ Process
Usage:
+ $ rm -f bugs # remove previously generated bug list
$ bin/review-bugs
3. Look at all uploads for the past week.
@@ -66,6 +76,17 @@ Process
<(git show "master@{$(LC_ALL=C date --date='a week ago' +'%b %d')}":packages.yml) \
<(git show "master@{$(LC_ALL=C date +'%b %d')}":packages.yml)
+ The UDD query to know who reported how many FTBFS follows:
+
+ SELECT bugs.submitter, count(distinct bugs.id)
+ FROM bugs_usertags, bugs
+ WHERE bugs_usertags.email = 'reproducible-builds at lists.alioth.debian.org'
+ AND bugs_usertags.tag = 'ftbfs'
+ AND bugs.id = bugs_usertags.id
+ AND bugs.arrival > DATE_TRUNC('DAY', NOW() - INTERVAL '7 DAY')
+ AND bugs.arrival < DATE_TRUNC('DAY', NOW())
+ GROUP BY bugs.submitter;
+
5. New and updated issues:
Make sure you get a recent copy of `notes.git`. Then:
diff --git a/reports/bin/newly-reproducible b/reports/bin/newly-reproducible
index fb86fda..86705bc 100755
--- a/reports/bin/newly-reproducible
+++ b/reports/bin/newly-reproducible
@@ -4,6 +4,7 @@
# Copyright © 2015 Lunar <lunar at debian.org>
# Licensed under WTFPL — http://www.wtfpl.net/txt/copying/
+import re
import sqlite3
conn = sqlite3.connect('reproducible.db')
@@ -22,7 +23,7 @@ for name, reproducible_version, suite, reproducible_build_time in c.execute('SEL
if res:
continue
for version, status, build_time in c2.execute('SELECT version, status, strftime("%s", build_date) FROM stats_build WHERE name = ? AND suite = ? AND build_date < DATETIME(?, "unixepoch") ORDER BY build_date DESC', (name, suite, int(reproducible_build_time) - 1)):
- if status in ('reproducible', 'FTBFS'):
+ if status in ('reproducible', 'FTBFS', 'depwait'):
if name in now_reproducible and 'toolchain' in now_reproducible[name]:
del now_reproducible[name]
continue
@@ -36,7 +37,7 @@ for name, reproducible_version, suite, reproducible_build_time in c.execute('SEL
prefix = name[0:4]
else:
prefix = name[0]
- changelog_url = 'https://tracker.debian.org/media/packages/%s/%s/changelog-%s' % (prefix, name, reproducible_version)
+ changelog_url = 'http://metadata.ftp-master.debian.org/changelogs/main/%(prefix)s/%(name)s/%(name)s_%(version)s_changelog' % { 'prefix':prefix, 'name': name, 'version': re.sub(r'^[0-9]+:', '', reproducible_version) }
now_reproducible[name] = 'since %s over %s <%s>' % (reproducible_version, version, changelog_url)
else:
print('UNKNOWN STATUS %s' % status)
diff --git a/reports/bin/review-bugs b/reports/bin/review-bugs
index 3f7f71a..6a1b926 100755
--- a/reports/bin/review-bugs
+++ b/reports/bin/review-bugs
@@ -4,11 +4,16 @@
# Copyright © 2015 Lunar <lunar at debian.org>
# Licensed under WTFPL — http://www.wtfpl.net/txt/copying/
-BUGS=$(echo "SELECT DISTINCT bugs.id FROM bugs_usertags, bugs WHERE email = 'reproducible-builds at lists.alioth.debian.org' AND bugs.id = bugs_usertags.id AND bugs.last_modified > NOW() - INTERVAL '7 DAY' ORDER BY bugs.id;" | ssh alioth.debian.org psql service=udd -t)
+if ! [ -f bugs ]; then
+ echo "SELECT DISTINCT bugs.id FROM bugs_usertags, bugs WHERE email = 'reproducible-builds at lists.alioth.debian.org' AND bugs.id = bugs_usertags.id AND bugs.last_modified > DATE_TRUNC('DAY', NOW() - INTERVAL '7 DAY') AND bugs.last_modified < DATE_TRUNC('DAY', NOW()) ORDER BY bugs.id;" | ssh alioth.debian.org psql service=udd -t | awk '/[0-9]/ { print $1 }' > bugs
+fi
-for bug in $BUGS; do
- bts cache --cache-mode=mbox "$bug"
-done
-for bug in $BUGS; do
+TOTAL="$(wc -l bugs | awk '{ print $1 }')"
+
+for bug in $(cat bugs); do
+ bts cache --cache-mode=mbox "$bug" >/dev/null
+ echo "$bug"
+done | pv -t -e -p -l -s "$TOTAL"
+for bug in $(cat bugs); do
bts -o show --mbox "$bug"
done
diff --git a/reports/template.mdwn b/reports/template.mdwn
index 92a7f25..3116c87 100644
--- a/reports/template.mdwn
+++ b/reports/template.mdwn
@@ -1,8 +1,8 @@
[[!meta title="Reproducible builds: week XXX in Stretch cycle"]]
[[!tag reproducible_builds]]
-What happened about the [reproducible
-builds](https://wiki.debian.org/ReproducibleBuilds) effort for this week:
+What happened in the [reproducible
+builds](https://wiki.debian.org/ReproducibleBuilds) effort this week:
Media coverage
--------------
@@ -22,12 +22,18 @@ The following packages became reproducible after getting fixed:
Some uploads fixed some reproducibility issues but not all of them:
-Patches submitted which did not make their way to the archive yet:
+Patches submitted which have not made their way to the archive yet:
reproducible.debian.net
-----------------------
-debbindiff development
+diffoscope development
+----------------------
+
+strip-nondeterminism development
+--------------------------------
+
+disorderfs development
----------------------
Documentation update
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/reproducible/misc.git
More information about the Reproducible-commits
mailing list