[Reproducible-commits] [presentations] 01/01: work in progress…
Holger Levsen
holger at moszumanska.debian.org
Wed Jan 27 19:38:16 UTC 2016
This is an automated email from the git hooks/post-receive script.
holger pushed a commit to branch master
in repository presentations.
commit 1f7aba95a9ef9594565bc9bc5e2b75e878300685
Author: Holger Levsen <holger at layer-acht.org>
Date: Wed Jan 27 18:58:10 2016 +0100
work in progress…
---
.../2016-01-31-FOSDEM16-Reproducible-ecosystem.tex | 318 ++++++++++-----------
2016-01-31-FOSDEM16/notes | 3 +
2 files changed, 155 insertions(+), 166 deletions(-)
diff --git a/2016-01-31-FOSDEM16/2016-01-31-FOSDEM16-Reproducible-ecosystem.tex b/2016-01-31-FOSDEM16/2016-01-31-FOSDEM16-Reproducible-ecosystem.tex
index 463b9a0..ffe142d 100644
--- a/2016-01-31-FOSDEM16/2016-01-31-FOSDEM16-Reproducible-ecosystem.tex
+++ b/2016-01-31-FOSDEM16/2016-01-31-FOSDEM16-Reproducible-ecosystem.tex
@@ -203,7 +203,7 @@ won't be enough}
\end{itemize}
\end{frame}
-\section{About}
+\section{Motivation}
\begin{frame}
\frametitle{The problem}
@@ -288,67 +288,10 @@ won't be enough}
\end{center}
\end{frame}
-\section{Progress}
-
-\begin{frame}[plain]
- \frametitle{Progress in Debian \texttt{unstable}}
- \begin{center}
- \includegraphics[height=0.73\paperheight]{images/stats_pkg_state.png}
-
- \footnotesize{19,257 out of 23,141 source packages are reproducible \\
- in our test framework}
- \vfill
- \end{center}
-\end{frame}
-
-
-\begin{frame}[plain]
- \frametitle{Progress in the Debian Bug Tracking System (BTS)}
- \begin{tikzpicture}[remember picture,overlay]
- \node[at=(current page.center)] {
- \includegraphics[height=0.73\paperheight]{images/stats_bugs_state.png}
- };
- \end{tikzpicture}
-\end{frame}
+\section{Common ressources}
\begin{frame}
- \frametitle{What we did in Debian since Summer 2014}
-
- \begin{itemize}
- \item Agreed on using a fixed build path: \texttt{/build/}
- \item Recording the build environment: \texttt{.buildinfo}
- \item \texttt{strip-nondeterminism}
- \item \texttt{reproducible.debian.net} has become
- \texttt{tests.reproducible-builds.org}
- \item \texttt{diffoscope} (formerly \texttt{debbindiff})
- \item \texttt{SOURCE\_DATE\_EPOCH}
- \item \texttt{disorderfs}
- \item 700+ patches: \texttt{dpkg}, \texttt{debhelper}, \texttt{sbuild}, …
- \item<2> Tell the world \& collaborate
- \end{itemize}
-\end{frame}
-
-
-\begin{frame}
- \frametitle{Tell the world \& collaborate}
-
- \begin{itemize}
- \item Recent talks (some available with subtitles):
- \begin{itemize}
- \item 2015-08-13: Chaos Communication Camp 2015
- \item 2015-08-20: DebConf15
- \item 2015-11-08: Mini-DebConf Cambridge 2015
- \end{itemize}
- \item Weekly reports since May 2015
- \item Summit in December 2015 (Athens)
- \begin{itemize}
- \item 40 people from 16 projects
- \end{itemize}
- \end{itemize}
-\end{frame}
-
-\begin{frame}
- \frametitle{Tell the world \& collaborate, cont.}
+ \frametitle{reproducible-builds.org}
\begin{itemize}
\item \texttt{https://reproducible-builds.org}
@@ -359,16 +302,15 @@ won't be enough}
\end{frame}
\begin{frame}
- \frametitle{Stats about tests.reproducible-builds.org}
+ \frametitle{tests.reproducible-builds.org}
\begin{itemize}
\item Continuously testing Debian testing, unstable and experimental
\begin{itemize}
\item \small{ \texttt main only }
- \item \small{ can we build \texttt contrib without legal troubles? }
\end{itemize}
\item Also testing coreboot, OpenWrt, NetBSD, FreeBSD,
- Archlinux and Fedora
+ Archlinux, Fedora and soon FDroid and Guix too
\begin{itemize}
\item \small{ those a bit less frequently though… }
\end{itemize}
@@ -381,15 +323,16 @@ won't be enough}
\begin{frame}
- \frametitle{More stats on tests.reproducible-builds.org}
+ \frametitle{More on tests.reproducible-builds.org}
\begin{itemize}
- \item 122 jenkins jobs running on 12 hosts
- \item 27 contributors for \texttt{jenkins.debian.net.git}
- \item 4k lines of Python and 5k lines Bash code
+ \item 230 jenkins jobs running on 22 hosts
+ \item 29 contributors for \texttt{jenkins.debian.net.git}
+ \item 4k lines of Python and 6k lines Bash code
\item \texttt{amd64}: 111 cores and 198 GB RAM split on 9 VMs, provided by
https://profitbricks.co.uk
- \item \texttt{armhf}: 18 cores and 9 GB RAM on 6 systems, provided by vagrant at d.o.
+ \item \texttt{armhf}: 50 cores and 23 GB RAM on 14 systems drawing 100 watts
+ (combined), provided by vagrant at debian.org.
\end{itemize}
\begin{center}
\includegraphics[height=0.2\paperheight]{images/profitbricks_logo.png}
@@ -397,40 +340,8 @@ won't be enough}
\end{center}
\end{frame}
-\begin{frame}
- \frametitle{Good to know about tests.reproducible-builds.org}
-
- \begin{itemize}
- \item \url {https://reproducible.debian.net/$src}
- \item<2-3> { 165 categorised distinct issues }
- \item<2-3> { 3,496 packages to be fixed in \texttt{sid}, but only 426 without annotated
- issues }
- \item<3> { 29 different "package sets", eg. \texttt{build-essential} is only 78\%
- reproducible
- \begin{center}
- \includegraphics[height=0.5\paperheight]{images/stats_meta_pkg_state_build-essential.png}
- \vfill
- \end{center}
- }
- \end{itemize}
-\end{frame}
-
-\begin{frame}
- \frametitle{Future of tests.reproducible-builds.org}
-
- \begin{itemize}
- \item We still want more arm(64) cores!
- \item We want to test on other architectures!
- \item We want more people looking at the results!
- \item We want more people contributing code for their projects!
- \item We don't want to build twice and test against what we built, but rather
- the binaries distributed by these projects (if any)
- \end{itemize}
-\end{frame}
-
-
\begin{frame}[fragile]
- \frametitle{Variations on reproducible.debian.net}
+ \frametitle{Variations on when testing Debian}
\begin{center}
\begin{table}
@@ -461,49 +372,38 @@ hour, minute & \multicolumn{2}{l}{hour is usually the same… usually, the minut
\end{frame}
-
\begin{frame}
- \frametitle{Debian \texttt{.buildinfo} files}
+ \frametitle{Good to know about tests.reproducible-builds.org}
\begin{itemize}
- \item Aggregates in the same file:
- \begin{itemize}
- \item Sources (checksums)
- \item Generated binaries (checksums)
- \item Packages used to build (with specific version, checksums coming soon)
- \end{itemize}
- \item Can be later used to exactly recreate environment
- \item For Debian, all versions are available from \url{snapshot.debian.org}
+ \item \url {https://reproducible.debian.net/$src}
+ \item<2-3> { 165 categorised distinct issues }
+ \item<2-3> { 3,496 packages to be fixed in \texttt{sid}, but only 426 without annotated
+ issues }
+ \item<3> { 29 different "package sets", eg. \texttt{build-essential} is only 78\%
+ reproducible
+ \begin{center}
+ \includegraphics[height=0.5\paperheight]{images/stats_meta_pkg_state_build-essential.png}
+ \vfill
+ \end{center}
+ }
\end{itemize}
\end{frame}
+\begin{frame}
+ \frametitle{Future of tests.reproducible-builds.org}
-\begin{frame}[fragile]
- \frametitle{Example \texttt{.buildinfo} file}
-
-{\small
-\begin{verbatim}
-Format: 1.9
-Build-Architecture: amd64
-Source: txtorcon
-Binary: python-txtorcon
-Architecture: all
-Version: 0.11.0-1
-Build-Path: /build/txtorcon-0.11.0-1
-Checksums-Sha256:
- a26549d9…7b 125910 python-txtorcon_0.11.0-1_all.deb
- 28f6bcbe…69 2039 txtorcon_0.11.0-1.dsc
-Build-Environment:
- base-files (= 8),
- base-passwd (= 3.5.37),
- bash (= 4.3-11+b1),
- …
-\end{verbatim}
-}
+ \begin{itemize}
+ \item We still want more arm(64) cores!
+ \item We want to test on other architectures!
+ \item We want more people looking at the results!
+ \item We want more people contributing code for their projects!
+ \item We don't want to build twice and test against what we built, but rather
+ the binaries distributed by these projects (if any)
+ \end{itemize}
\end{frame}
-
{
\usebackgroundtemplate{%
\begin{tikzpicture}[remember picture,overlay]%
@@ -512,6 +412,8 @@ Build-Environment:
};
\end{tikzpicture}%
}
+
+
\begin{frame}{diffoscope}
\frametitle{Debugging problems: diffoscope}
@@ -580,9 +482,6 @@ Build-Environment:
\end{frame}
-\section{Next steps}
-
-
\begin{frame}
\frametitle{\texttt{SOURCE\_DATE\_EPOCH} (open bugs)}
@@ -602,6 +501,22 @@ Build-Environment:
\end{frame}
+
+\section{Status Debian}
+
+\begin{frame}[plain]
+ \frametitle{Progress in Debian \texttt{unstable}}
+ \begin{center}
+ \includegraphics[height=0.73\paperheight]{images/stats_pkg_state.png}
+
+ \footnotesize{19,946 out of 23,584 source packages are reproducible \\
+ in our test framework}
+ \vfill
+ \end{center}
+\end{frame}
+
+
+
\begin{frame}
\frametitle{Missing bits}
\begin{itemize}
@@ -611,29 +526,105 @@ Build-Environment:
\end{itemize}
\end{frame}
+\begin{frame}[plain]
+ \frametitle{Progress in the Debian Bug Tracking System (BTS)}
+ \begin{tikzpicture}[remember picture,overlay]
+ \node[at=(current page.center)] {
+ \includegraphics[height=0.73\paperheight]{images/stats_bugs_state.png}
+ };
+ \end{tikzpicture}
+\end{frame}
+
\begin{frame}
- \frametitle{\texttt{dpkg}}
+ \frametitle{What we did in Debian since Summer 2014}
\begin{itemize}
- \item \sout{\texttt{\#719844}: make compression of \{data,control\}.tar.gz deterministic}
- \item \texttt{\#759999}: set reproducible timestamps in \texttt{.deb} ar file headers
- \item \texttt{\#787980}: normalize file permissions when creating control.tar
- \item \texttt{\#719845}: make file order within {data,control}.tar.gz deterministic
- \item \texttt{dpkg-genbuldinfo}: \textit{patch already written, but waiting on agreement about spec}
+ \item Agreed on using a fixed build path: \texttt{/build/}
+ \item Recording the build environment: \texttt{.buildinfo}
+ \item \texttt{strip-nondeterminism}
+ \item \texttt{reproducible.debian.net} has become
+ \texttt{tests.reproducible-builds.org}
+ \item \texttt{diffoscope} (formerly \texttt{debbindiff})
+ \item \texttt{SOURCE\_DATE\_EPOCH}
+ \item \texttt{disorderfs}
+ \item 700+ patches: \texttt{dpkg}, \texttt{debhelper}, \texttt{sbuild}, …
+ \item …
\end{itemize}
\end{frame}
+
\begin{frame}
- \frametitle{\texttt{debhelper}}
+ \frametitle{Tell the world \& collaborate}
\begin{itemize}
- \item \texttt{\#759886}: make mtimes of packaged files deterministic
- \item \sout{\texttt{\#759895}: add a call to
- \texttt{dh\_strip\_nondeterminism} in \texttt{dh}}
- \item \sout{\texttt{\#791823}: set \texttt{SOURCE\_DATE\_EPOCH} env var for
- reproducible builds}
- \item \sout{\texttt{\#802005}: sort file lists passed to 'cp --parents -p'
- for reproducibility}
+ \item Recent talks (some available with subtitles):
+ \begin{itemize}
+ \item 2015-08-13: Chaos Communication Camp 2015
+ \item 2015-08-20: DebConf15
+ \end{itemize}
+ \item Weekly reports since May 2015
+ \item Summit in December 2015 (Athens)
+ \begin{itemize}
+ \item 40 people from 16 projects
+ \end{itemize}
+ \end{itemize}
+\end{frame}
+
+
+
+
+\begin{frame}
+ \frametitle{Debian \texttt{.buildinfo} files}
+
+ \begin{itemize}
+ \item Aggregates in the same file:
+ \begin{itemize}
+ \item Sources (checksums)
+ \item Generated binaries (checksums)
+ \item Packages used to build (with specific version, checksums coming soon)
+ \end{itemize}
+ \item Can be later used to exactly recreate environment
+ \item For Debian, all versions are available from \url{snapshot.debian.org}
+ \end{itemize}
+\end{frame}
+
+
+\begin{frame}[fragile]
+ \frametitle{Example \texttt{.buildinfo} file}
+
+{\small
+\begin{verbatim}
+Format: 1.9
+Build-Architecture: amd64
+Source: txtorcon
+Binary: python-txtorcon
+Architecture: all
+Version: 0.11.0-1
+Build-Path: /build/txtorcon-0.11.0-1
+Checksums-Sha256:
+ a26549d9…7b 125910 python-txtorcon_0.11.0-1_all.deb
+ 28f6bcbe…69 2039 txtorcon_0.11.0-1.dsc
+Build-Environment:
+ base-files (= 8),
+ base-passwd (= 3.5.37),
+ bash (= 4.3-11+b1),
+ …
+\end{verbatim}
+}
+\end{frame}
+
+
+
+
+\begin{frame}
+ \frametitle{\texttt{dpkg}}
+
+ \begin{itemize}
+ \item \sout{\texttt{\#719844}: make compression of \{data,control\}.tar.gz deterministic}
+ \item \texttt{\#759999}: set reproducible timestamps in \texttt{.deb} ar file headers
+ \item \texttt{\#787980}: normalize file permissions when creating control.tar
+ \item \texttt{\#719845}: make file order within {data,control}.tar.gz deterministic
+ \item \texttt{dpkg-genbuldinfo}: \textit{patch already written, but waiting on agreement about spec}
\end{itemize}
\end{frame}
@@ -667,7 +658,7 @@ Build-Environment:
\end{itemize}
\end{frame}
-\section{Beyond building}
+\section{Future work}
\begin{frame}
\frametitle{Reproducible builds demand a defined build environment}
@@ -737,19 +728,10 @@ Build-Environment:
\end{itemize}
\end{frame}
-\begin{frame}
- \frametitle{Integration in user tools - conclusion}
- \begin{itemize}
- \item "Rebuilders and sharing signed checksums" needs to be designed
- (and probably at least partly implemented) before thinking more about
- user tools. It's just clear we need them.
- \end{itemize}
-\end{frame}
-
\section{Want to help?}
\begin{frame}
- \frametitle{As a developer}
+ \frametitle{As a software developer}
\begin{itemize}
\item Stop using build dates
\item Use \texttt{SOURCE\_DATE\_EPOCH} instead
@@ -776,7 +758,7 @@ Build-Environment:
\end{frame}
\begin{frame}
- \frametitle{Join the team!}
+ \frametitle{Join the (Debian) team!}
\begin{itemize}
\item Why?
@@ -798,7 +780,7 @@ Build-Environment:
\end{frame}
\begin{frame}
- \frametitle{Create a new team!}
+ \frametitle{Form another team!}
\begin{itemize}
\item Why?
@@ -806,6 +788,8 @@ Build-Environment:
\item Every distribution should be reproducible!
\item Learn something new everyday
\item Change the (software) world!
+ \item \texttt{https://tests.reproducible-builds.org/\$project} needs
+ \textbf{your} help
\end{itemize}
\item How to get started?
\begin{itemize}
@@ -861,10 +845,12 @@ Build-Environment:
\vfill
\begin{center}
- \resizebox{0.8\textwidth}{!}{%
+ \resizebox{0.9\textwidth}{!}{%
\begin{tabular}{rl}
\texttt{holger at debian.org} & \texttt{B8BF 5413 7B09 D35C F026} \\
- & \texttt{FE9D 091A B856 069A AA1C}
+ & \texttt{FE9D 091A B856 069A AA1C} \\
+ \\
+ \texttt{https://reproducible-builds.org}
\end{tabular}
}
\end{center}
diff --git a/2016-01-31-FOSDEM16/notes b/2016-01-31-FOSDEM16/notes
index d469573..b87c687 100644
--- a/2016-01-31-FOSDEM16/notes
+++ b/2016-01-31-FOSDEM16/notes
@@ -43,6 +43,9 @@ www.reproducible-builds.org
/specs / SOURCE_DATE_EPOCH
tests.r-b.o today (are new - used to be r.d.n)
tests.r-b.o future plans (repeat future plan before thanks)
+diffoscope
+
+add SOURCE_DATE_EPOCH adoption outside Debian…
mention athen meeting somewhere appropriate
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/reproducible/presentations.git
More information about the Reproducible-commits
mailing list