[buildd-tools-devel] Wanted: system/volunteer to do whole-archive rebuild (×3) to test sbuild
Roger Leigh
rleigh at codelibre.net
Mon Feb 21 08:04:51 UTC 2011
On Mon, Feb 14, 2011 at 11:36:55PM +0000, Roger Leigh wrote:
> sbuild, which does the job of building binary packages on our buildds,
> uses a built-in build-dependency resolver ("internal") to work out
> what packages need installing/removing in order to satisfy a package's
> Build-Depends and Build-Conflicts. Unfortunately, this has a number
> of bugs, e.g. #403246, as well as serious maintainability issues which
> make it less than desirable. Last year, we introduced two new
> resolvers, "apt" and "aptitude" which delegate the somewhat complex
> build dependency resolving to the tools which are best at it.
>
> Now that squeeze is out, it would be great if we could revisit the
> discussion about which build dependency resolver we want to use on
> our buildds. The main concern here is that switching resolvers will
> change exactly which packages are installed in some situations, mainly
> in the case of packages depending on alternatives, which could lead to
> different packages being installed, inconsistent selection of packages
> being installed, or broken package builds. The intenal resolver was
> dumb: it just always chose the first dependency even if it was
> unsatisfiable. aptitude is far cleverer; apt somewhere in between,
> though we've tweaked aptitude to behave as close to stupid as we can.
The results of this are below. Many thanks to Tollef Fog Heen for
his kind donation of time on one of his big machines to run three
archive rebuilds. A better quality PDF version is attached.
Due to the size limits of the lists, the full version is at
http://people.debian.org/~rleigh/squeeze-rebuild/report.pdf
Regards,
Roger
sbuild Build Dependency Resolver Comparison
1. Introduction
sbuild has three different build dependency resolvers,
internal, apt and aptitude. The historical default is
internal, while apt and aptitude are much newer. The inter‐
nal resolver is a build‐dependency resolver implemented
directly in Perl, and which has a number of deficiencies
mainly relating to newer additions to the dependency format,
and also in resolving of complex versioned and virtual
dependencies. The apt and aptitude resolvers delegate
almost all dependency resolution to the apt and aptitude
tools, whose dependency resolution is far more capable and
better tested. However, the strict, simple, predictable
dependency resolution provided by internal must also be pro‐
vided by the alternative resolvers in order for them to be
suitable for use on our build dæmons.
A goal for the wheezy release is to replace the inter‐
nal resolver with one of the apt or aptitude resolvers.
However, we had no hard information upon which to base a
decision. I have consequently rebuilt the entire Debian ar‐
chive three times using the internal, apt and aptitude
resolvers, using the squeeze release since it’s stable and
unchanging between the runs. The build logs have been pro‐
cessed and the build status and dependency information
inserted into a PostgreSQL database in order to allow direct
comparison of the behaviour of the three resolvers.
These tests all used the current version of sbuild from
unstable (0.60.9‐1), with a one line patch to the aptitude
resolver to make it use the same dpkg options as the other
resolvers when installing packages. The internal resolver
has a bugfix the version on the buildds does not yet have,
allowing it to delegate virtual dependency resolution to
apt‐get rather than failing outright (since older versions’
handling of virtual packages was completely broken).
Many thanks to Tollef Fog Heen and freedesktop.org for
the use of a big machine on which to do the builds. This 24
core beast was able to rebuild the entire archive in 14
hours, taking four days in total for the three builds.
The build logs, database schema and database dump are
all available at http://people.debian.org/~rleigh/squeeze‐
rebuild/.
2. Build summary
These are the summary statistics for all the builds:
┌─────────┬────────────┬──────────────┬───────┐
│Resolver │ Status │ Fail stage │ Count │
├─────────┼────────────┼──────────────┼───────┤
│apt │ attempted │ build │ 38 │
│apt │ given‐back │ install‐deps │ 5 │
│apt │ skipped │ arch‐check │ 6195 │
│apt │ successful │ │ 8372 │
├─────────┼────────────┼──────────────┼───────┤
│aptitude │ attempted │ build │ 41 │
│aptitude │ skipped │ arch‐check │ 6195 │
│aptitude │ successful │ │ 8374 │
├─────────┼────────────┼──────────────┼───────┤
│internal │ attempted │ build │ 37 │
│internal │ given‐back │ install‐deps │ 16 │
│internal │ skipped │ arch‐check │ 6195 │
│internal │ successful │ │ 8362 │
└─────────┴────────────┴──────────────┴───────┘
Skipped packages failing the architecture check are
Architecture: all packages; these were all skipped by the
different resolvers prior to installation of build dependen‐
cies, and will not be considered further. Packages were
given back due to errors during installation of build depen‐
dencies, and comprised:
┌─────────┬───────────────────────────────────────────┐
│Resolver │ Build │
├─────────┼───────────────────────────────────────────┤
│apt │ circlepack_5.1‐7 │
│apt │ libcomplearn‐mod‐ppmd_1.0.7‐2 │
│apt │ libcomplearn‐mod‐ppmdx_1.0.7‐2 │
│apt │ linux‐modules‐di‐amd64‐2.6_1.23 │
│apt │ xserver‐xorg‐video‐glide_1.0.3‐2+squeeze1 │
├─────────┼───────────────────────────────────────────┤
│internal │ allegro4.2_2:4.2.2‐2.2 │
│internal │ arts_1.5.9‐3 │
│internal │ cheesetracker_0.9.15.3‐4 │
│internal │ circlepack_5.1‐7 │
│internal │ djplay_0.5.0‐3.1 │
│internal │ dovecot_1:1.2.15‐4 │
│internal │ ecasound2.2_2.7.0‐1.1 │
│internal │ galan_0.3.0+beta4‐2 │
│internal │ libcomplearn‐mod‐ppmd_1.0.7‐2 │
│internal │ libcomplearn‐mod‐ppmdx_1.0.7‐2 │
│internal │ libvisual‐plugins_0.4.0.dfsg.1‐2 │
│internal │ linux‐modules‐di‐amd64‐2.6_1.23 │
│internal │ specimen_0.5.2rc3‐1.1 │
│internal │ t38modem_1.2.0‐1 │
│internal │ timemachine_0.3.0‐3 │
│internal │ xserver‐xorg‐video‐glide_1.0.3‐2+squeeze1 │
└─────────┴───────────────────────────────────────────┘
circlepack, libcomplearn‐mod‐ppmd, libcomplearn‐mod‐
ppmdx, linux‐modules‐di‐amd64 and xserver‐xorg‐video‐glide
were not built by either the apt or internal resolvers
because the needed build dependencies were unavailable.
They did build not build with aptitude either, but the apti‐
tude failure did not terminate the build; these are reported
as attempted builds because failure occurs when dpkg‐build‐
package notices the build dependencies are not satisfied.
This will require fixing in the aptitude resolver.
allegro, arts, cheesetracker, djplay, ecasound2, galan,
libvisual‐plugins, specimen and timemachine all have build
dependencies upon libjack_0.100.0‐dev which is a virtual
package. The internal resolver can not resolve virtual
dependencies, while the other resolvers can. These packages
should be fixed to use libjack‐dev.
dovecot (bug filed), steghide (already fixed in unsta‐
ble) and t38modem (bug filed) have a versioned build con‐
flict upon linux‐kernel‐headers which causes the internal
resolver to purge build‐essential and the whole toolchain,
which causes build failure. This isn’t a problem with the
build‐conflict being incorrect, though a conflict on linux‐
kernel‐headers (<< 2.5.999‐test7‐bk‐14) is certainly useless
and outdated, this is a bug in the internal resolver. How‐
ever, the build conflict is outdated; given that we only
support linux >= 2.6, so it can be safely removed. The more
advanced apt and aptitude resolvers could cope with the ver‐
sioned virtual conflict.
Attempted (failed) builds failed during building with
dpkg‐buildpackage, and were the same for all resolvers,
allowing for differences in dependency resolution failure.
3. Methodology
In order to allow for meaningful comparisons between
resolvers, only builds which were successful with all three
resolvers will be compared in the following sections. The
apt and aptitude resolvers install additional packages in
order to function, which require excluding from the compar‐
isons. Both the apt and aptitude resolvers install dummy
dependency packages in order to delegate resolving to the
apt or aptitude programs, which should not be included in
the differences between the resolvers. Additionally, the
aptitude resolver requires aptitude, and its package depen‐
dencies, to be installed, and these also require exclusion.
By removing these known differences, only the unknown dif‐
ferences will show up.
Using SQL set difference (EXCEPT), the packages
installed only when using the internal resolver, or only
when using the apt (or apt resolver) can be compared; pack‐
ages installed in both situations will be ignored. NOT IN
will be used to remove unwanted resolver and dependency
packages. The database schema, views and queries used to
analyse the data, together with the scripts used to perform
the builds and parse the build logs to get the data to
insert into the database are in the appendix at the end of
this document.
4. Differences between internal and apt
A total of 54 out of 8345 successful builds (0.65%)
showed discrepancies in the installed package set.
[Detail removed]
5. Differences between internal and aptitude
A total of 55 out of 8345 successful builds (0.66%)
showed discrepancies in the installed package set.
[Detail removed]
6. Differences between apt and aptitude
The package sets installed by apt and aptitude were
almost identical. Only two packages out of 8345 successful
builds (0.02%) showed discrepancies in the installed package
set.
Packages installed with internal and apt (not apti‐
tude):
┌────────────────┬───────────────────────────┐
│Build │ Package │
└────────────────┴───────────────────────────┘
│tulip_3.1.2‐2.3 │ libdrm2_2.4.21‐1~squeeze3 │
│ │ libxdamage1_1:1.1.3‐1 │
│ │ libxfixes3_1:4.0.5‐1 │
│ │ libxxf86vm1_1:1.1.0‐2 │
└────────────────┴───────────────────────────┘
Packages installed with internal and aptitude (not
apt):
┌────────────────┬──────────────────────┐
│Build │ Package │
└────────────────┴──────────────────────┘
│piklab_0.15.7‐1 │ libsqlite3‐0_3.7.3‐1 │
└────────────────┴──────────────────────┘
7. Analysis of resolver differences
alex_2.3.3‐1
apt and aptitude install docbook_4.5‐4.
Depends on docbook‐utils, docbook‐xsl and docbook‐xml.
docbook is only suggested, and there is an alternative
dependency in docbook‐dsssl. apt‐get and aptitude behave
differently when installing a dependency via the dummy pack‐
age than they do when invoked directly as internal does.
When invoked with a list of packages to install, docbook is
not installed, presumably because the alternative dependency
is already satisfied.
avogadro_1.0.1‐3
internal installs sip4_4.10.2‐1.
sip4 is a transitional package. apt and aptitude are
intelligent enough to realise that this is provided by
another package and not install it, whereas internal is not.
boxbackup_0.11~rc2‐7squeeze1
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
calibre_0.7.7+dfsg‐1squeeze1
internal installs sip4_4.10.2‐1.
Same reason as for avogadro.
coinor‐cbc_2.5.0‐2
apt and aptitude install libatlas‐base‐dev_3.8.3‐27 and
libatlas‐dev_3.8.3‐27.
This is an alternative dependency in liblapack‐dev. It
can be satisfied by libblas‐3gf.so provided by libblas‐dev.
internal doesn’t pick both alternatives, while apt and
internal do.
ecj_3.5.1‐1
internal installs ca‐certificates_20090814+nmu2, ca‐
certificates‐java_20100412, default‐jre‐headless_1:1.6‐40,
liblcms1_1.18.dfsg‐1.2+b3, libnspr4‐0d_4.8.6‐1, lib‐
nss3‐1d_3.12.8‐1, openjdk‐6‐jre‐headless_6b18‐1.8.3‐2, open‐
jdk‐6‐jre‐lib_6b18‐1.8.3‐2, openssl_0.9.8o‐4 and tzdata‐
java_2010o‐1
This is a redundant set of packages pulled in via the
ant build dependency, which has a complex set of four possi‐
ble alternatives. apt and aptitude realise that this is
satisfied by gcj‐4.4‐jdk, and do not install them.
elinks_0.12~pre5‐2
internal installs docbook_4.5‐4.
The build dependency docbook‐utils depends upon doc‐
book‐dsssl which in turn depends upon either docbook or doc‐
book‐xml. When run via apt‐get or aptitude directly, doc‐
book is installed, and when installed via a dependency pack‐
age, it is not.
enblend‐enfuse_4.0+dfsg‐1
apt and aptitude install ttf‐dejavu‐core_2.31‐1.
The fontconfig‐config dependency has alternative depen‐
dencies upon four different font packages. internal
realises that they are already satisfied, whereas apt and
aptitude do not.
f‐spot_0.6.2‐2
apt and aptitude install automake_1:1.11.1‐1.
The build dependency intltool has an alternative depen‐
dency on automake or automaken. Consequently, apt and apti‐
tude install two versions of automake, and internal only
one.
gdcm_2.0.16‐2
apt and aptitude install gcj‐4.4‐base_4.4.5‐2,
gcj‐4.4‐jre_4.4.5‐2, gcj‐4.4‐jre‐headless_4.4.5‐2, gcj‐
jre_4:4.4.5‐1, gcj‐jre‐headless_4:4.4.5‐1, libgcj10_4.4.5‐2,
libgcj10‐awt_4.4.5‐2 and libgcj‐common_1:4.4.5‐1.
Possible alternatives affecting this are default‐jdk
and libvtk‐java. This is again a case of choosing multiple
alternatives.
gjiten_2.6‐2.1
internal installs docbook_4.5‐4.
Same reason as for elinks.
gmime2.2_2.2.25‐2
internal installs libosp5_1.5.2‐8,
libostyle1c2_1.4devel1‐19 and openjade_1.4devel1‐19.
Multiple docbook possibilities via alternatives in doc‐
book‐dsssl and gtk‐doc‐tools.
gmime2.4_2.4.14‐1+nmu1
internal installs libosp5_1.5.2‐8,
libostyle1c2_1.4devel1‐19 and openjade_1.4devel1‐19.
Same reasons as for gmime2.2.
gnome‐color‐manager_2.30.2‐1
internal installs libosp5_1.5.2‐8,
libostyle1c2_1.4devel1‐19 and openjade_1.4devel1‐19.
Same reasons as for gmime2.2.
gridengine_6.2u5‐1
internal installs default‐jre‐headless_1:1.6‐40.
Same reasons as for ecj.
happy_1.18.4‐2
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
hardware‐monitor_1.4.2‐1.1
apt and aptitude install makedev_2.3.1‐89.
Alternative dependency in libsensors4 build dependency.
hplip_3.10.6‐2
apt and aptitude install udev_164‐3, while internal
installs makedev_2.3.1‐89.
Same reason as for hardware‐monitor.
iceweasel_3.5.16‐4
apt and aptitude install ttf‐dejavu‐core_2.31‐1.
Same reason as for enblend‐enfuse.
idzebra_2.0.44‐2
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
java3d_1.5.2+dfsg‐5
internal installs default‐jre‐headless_1:1.6‐40.
Same reasons as for ecj.
jblas_1.1.1‐2
internal installs default‐jre‐headless_1:1.6‐40.
Same reasons as for ecj.
k3d_0.8.0.2‐6
apt and aptitude install ttf‐dejavu‐core_2.31‐1.
Same reason as for enblend‐enfuse.
libjdic‐java_0.9.5‐7
apt and aptitude install default‐jre‐headless_1:1.6‐40.
Same reasons as for ecj.
libjogl‐java_1.1.1+dak1‐9
internal installs default‐jre‐headless_1:1.6‐40.
Same reasons as for ecj.
libraw1394_2.0.5‐2
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
nut_2.4.3‐1.1squeeze1
apt and aptitude install makedev_2.3.1‐89.
Same reasons as for hardware‐monitor.
openmeeg_2.0.0.dfsg‐2
apt and aptitude install liblapack3gf_3.2.1‐8.
Same reasons as for coinor‐cbc.
openoffice.org_1:3.2.1‐11+squeeze2
internal installs default‐jre‐headless_1:1.6‐40.
Same reasons as for ecj.
opensp_1.5.2‐8
internal installs docbook_4.5‐4.
Same reason as for elinks.
openturns_0.13.2‐8
apt and aptitude install libatlas3gf‐base_3.8.3‐27,
libatlas‐base‐dev_3.8.3‐27 and libatlas‐dev_3.8.3‐27.
Same reasons as for coinor‐cbc.
ounit_1.0.3‐5
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
pauker_1.8+dfsg‐4
apt and aptitude install default‐jre_1:1.6‐40.
Same reasons as for ecj.
phaseshift_0.40‐13.2
internal installs xlibmesa‐gl‐dev_1:7.5+8.
This package is transitional. apt and aptitude are
clever enough to skip it, while internal uses the first
dependency.
php5_5.3.3‐7
internal installs libdb‐dev_4.8.
Another transitional‐like package where apt and apti‐
tude pick the direct dependency rather than using a meta‐
package.
piklab_0.15.7‐1
internal and aptitude install libsqlite3‐0_3.7.3‐1.
internal installs libasyncns0_0.3‐1.1, lib‐
cap2_1:2.19‐3, libflac8_1.2.1‐2+b1,
libphonon4_4:4.6.0really4.4.2‐1, libpulse0_0.9.21‐3+b1,
libpulse‐mainloop‐glib0_0.9.21‐3+b1, libqt4‐assis‐
tant_4:4.6.3‐4, libqt4‐dbus_4:4.6.3‐4,
libqt4‐designer_4:4.6.3‐4, libqt4‐dev_4:4.6.3‐4,
libqt4‐help_4:4.6.3‐4, libqt4‐multimedia_4:4.6.3‐4,
libqt4‐network_4:4.6.3‐4, libqt4‐qt3support_4:4.6.3‐4,
libqt4‐script_4:4.6.3‐4, libqt4‐scripttools_4:4.6.3‐4,
libqt4‐sql_4:4.6.3‐4, libqt4‐svg_4:4.6.3‐4,
libqt4‐test_4:4.6.3‐4, libqt4‐webkit_4:4.6.3‐4,
libqt4‐xml_4:4.6.3‐4, libqt4‐xmlpatterns_4:4.6.3‐4, libqt‐
core4_4:4.6.3‐4, libqtgui4_4:4.6.3‐4, libsndfile1_1.0.21‐3,
libwrap0_7.6.q‐19, libxtst6_2:1.1.0‐3 and
qt4‐qmake_4:4.6.3‐4
The package has an (arguably buggy) alternative build
dependency on both libqt4‐dev or libqt3‐mt‐dev. internal
installs both sets; apt and aptitude choose to only install
a single set. libsqlite3‐0 is pulled in via a very indirect
dependency, probably somewhere within the Qt libraries.
plplot_5.9.5‐4
apt and aptitude install ttf‐dejavu‐core_2.31‐1.
Same reason as for enblend‐enfuse.
poker‐network_1.7.7‐3.2
internal installs apache2.2‐bin_2.2.16‐6,
apache2.2‐common_2.2.16‐6, apache2‐mpm‐prefork_2.2.16‐6,
apache2‐utils_2.2.16‐6, libapache2‐mod‐php5_5.3.3‐7,
libapr1_1.4.2‐6, libaprutil1_1.3.9+dfsg‐5, libaprutil1‐dbd‐
sqlite3_1.3.9+dfsg‐5, libaprutil1‐ldap_1.3.9+dfsg‐5, lib‐
cap2_1:2.19‐3, libldap‐2.4‐2_2.4.23‐7, lib‐
sasl2‐2_2.1.23.dfsg1‐7 and procps_1:3.2.8‐9
Incredibly, these are all pulled in via libapache2‐mod‐
php5 through one of the php5 build dependencies (I still
haven’t fathomed which).
qmtest_2.4.1‐1
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
qtiplot_0.9.8‐1
internal installs docbook_4.5‐4, python‐
sip4‐dev_4.10.2‐1 and sip4_4.10.2‐1.
Same reason as for elinks for docbook, and as for avo‐
gadro for sip4.
rhythmbox_0.12.8‐3
libosp5_1.5.2‐8, libostyle1c2_1.4devel1‐19 and open‐
jade_1.4devel1‐19.
Same reasons as for gmime2.2.
rss‐glx_0.9.1‐3
internal installs libdrm2_2.4.21‐1~squeeze3 and
libxxf86vm1_1:1.1.0‐2.
This package has a number of alternative OpenGL build
dependencies. internal will only ever pick libgl1‐mesa‐
swx11‐dev, while apt and aptitude may pick one of the others
which does not include these packages.
scalapack_1.8.0‐6
apt and aptitude install libatlas3gf‐base_3.8.3‐27,
libatlas‐base‐dev_3.8.3‐27 and libatlas‐dev_3.8.3‐27.
Same reasons as for coinor‐cbc.
scidavis_0.2.4‐1
internal installs python‐sip4‐dev_4.10.2‐1.
Same reason as for avogadro.
shared‐mime‐info_0.71‐4
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
subversion_1.6.12dfsg‐4
internal installs ca‐certificates‐java_20100412,
default‐jre_1:1.6‐40, default‐jre‐headless_1:1.6‐40, libac‐
cess‐bridge‐java_1.26.2‐5, libaccess‐bridge‐java‐
jni_1.26.2‐5, libnspr4‐0d_4.8.6‐1, libnss3‐1d_3.12.8‐1,
openjdk‐6‐jre_6b18‐1.8.3‐2, openjdk‐6‐jre‐head‐
less_6b18‐1.8.3‐2, openjdk‐6‐jre‐lib_6b18‐1.8.3‐2 and
tzdata‐java_2010o‐1.
Multiple alternatives via junit.
taskjuggler_2.4.3‐2
apt and aptitude install opensp_1.5.2‐8.
There are a number of DocBook build dependencies con‐
taining alternative dependencies leading to multiple solu‐
tions.
thuban_1.2.2‐2
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
tig_0.16‐2
internal installs docbook_4.5‐4.
Same reason as for elinks.
trackballs_1.1.4‐4
internal installs xlibmesa‐gl‐dev_1:7.5+8.
Package contains an alternative build dependency upon
xlibmesa‐gl‐dev and libgl‐dev.
ttt_1.7‐3.2
internal installs fontconfig‐config_2.8.0‐2.1, libex‐
pat1_2.0.1‐7, libfontconfig1_2.8.0‐2.1,
libfreetype6_2.4.2‐2.1, libxext6_2:1.1.2‐1,
libxft2_2.1.14‐2, libxrender1_1:0.9.6‐1, libxss1_1:1.2.0‐2,
tcl8.5_8.5.8‐2, tk8.5_8.5.8‐1, ttf‐dejavu‐core_2.31‐1 and
ucf_3.0025+nmu1.
The build dependency blt‐dev depends on both versions
8.4 and 8.5 of tcl and tk. The internal resolver decides to
install both versions, whereas apt and aptitude do not.
tulip_3.1.2‐2.3
internal and apt install libdrm2_2.4.21‐1~squeeze3,
libxdamage1_1:1.1.3‐1, libxfixes3_1:4.0.5‐1 and
libxxf86vm1_1:1.1.0‐2.
Multiple alternative dependencies in build dependency
libqt4‐opengl‐dev and libosmesa6‐dev.
xmakemol_5.16‐5
internal installs libdrm2_2.4.21‐1~squeeze3, libxdam‐
age1_1:1.1.3‐1, libxfixes3_1:4.0.5‐1 and
libxxf86vm1_1:1.1.0‐2.
Multiple possible OpenGL solutions, as for previous
examples.
xplanet_1.2.1‐3 apt and aptitude install ttf‐dejavu‐
core_2.31‐1.
Same reason as for enblend‐enfuse.
yaz_4.0.11‐1
apt and aptitude install docbook_4.5‐4.
Same reason as for alex.
8. Identified differences between resolvers
The differences observed in all of the packages in the
previous section have the same root cause. The method of
installing build dependencies
internal invokes apt‐get with the argument install and
then a list of dependencies to install, then repeats this
with remove with a list of conflicts to remove. apt and
aptitude invoke apt‐get or aptitude, respectively, with an
install argument and a dummy dependency package containing
the complete set of depends and conflicts. Both apt‐get and
aptitude behave slightly differently if the package list is
provided on the command line or as a dependency package.
The build dependencies requested will be installed, but
depending upon indirect alternative dependencies, there may
be additional packages installed (or not) depending upon the
solution found. The alex and elinks packages, above, demon‐
strates this with each having two alternative solutions for
the DocBook dependencies. Note that neither solution is
incorrect; both are technically correct solutions.
The same different behaviour when faced with multiple
alternatives applies to all the other packages. Only a cou‐
ple of cases are directly due to alternative dependencies
directly in the build dependencies; all other cases are
indirect, in the dependencies of dependencies.
· The OpenGL library dependencies are overly complex;
these could use some simplification.
· The same situation exists with the multiple Java alter‐
natives, and differing ordering in different packages.
Again, some simplification is in order.
· And the same situation exists for DocBook; there are
just too many solutions to give consistent results.
The apt and aptitude resolvers, where they do differ
from the internal resolver, are not installing any different
build‐dependencies than which the maintainer requested. The
variation lies in leaf packages; the packages the maintainer
wanted are always installed.
9. Next steps
Is resolving virtual dependencies bad? Well, we are
already resolving them anyway. Any indirect virtual depen‐
dencies (virtual dependencies of build dependencies) are
being satisfied by apt‐get using the current internal
resolver. Supporting them in sbuild merely makes this be‐
haviour consistent, and as the above analysis demonstrates,
most of the alternatives are not in the build dependencies
themselves, they are indirect.
Are alternative dependencies bad? Almost unequivocally
yes. The entire point of build dependencies is to specify
exactly the packages required for building; by allowing mul‐
tiple possibilities, the reproducibility and consistency of
builds is compromised. Almost every single existing use of
alternative dependencies is buggy. There are only a limited
number of legitimate uses for alternative dependencies,
examples of which include using different packages on dif‐
ferent architectures (e.g. linux vs. kfreebsd vs. hurd)
where consistency on a single architecture is still guaran‐
teed, or to permit easier backporting when the packages have
been renamed. Every example seen here, the Qt 3 and 4
alternative being the worst, was buggy. However, the hand‐
ful of packages misusing alternatives are a tiny minority.
Whether or not alternative dependencies should be
allowed in build dependencies is a long‐standing point of
contention. The question of whether they are allowed is
completely orthogonal to whether they are possible. The apt
and aptitude resolvers make the use of alternative build
dependencies possible, as a side effect of using a fully
functional resolver. This does not imply that they should
be used. We should not enforce an unwritten policy through
long‐standing defects in our tools; the restrictions should
be defined in Policy, and compliance with Policy validated
with tools such as lintian.
To be completely clear about this, the purpose of the
apt and aptitude resolvers is not to enable the use of
alternative build dependencies. This is just a side effect.
The purpose is to replace a long obsolete and buggy resolver
with one which works under all circumstances, and doesn’t
break on a multitude of corner cases, and with new additions
to the dependency syntax. It will mean new syntax will be
usable when support is added to apt or aptitude, rather than
waiting for years, because the internal resolver is mostly
unmaintainable. The main gain is removing a big source of
unnecessary buggy complexity and replacing it with something
simple, understandable, maintainable and vastly more power‐
ful.
Is it safe to switch to apt or aptitude as the default
resolver? Almost certainly. This exhaustive test
demonstrates that 99.34–99.35% of the existing archive
builds entirely unchanged. Of the 0.65–0.66% which built
with an altered installed package set, zero failed to build.
The differences are explained by the different ways apt and
aptitude compute the dependencies depending upon how they
are invoked, and are almost entirely benign. There are some
packages with currently buggy dependencies which should be
fixed irrespective of the resolver used. The apt and apti‐
tude build resolvers are largely equivalent, with only a
0.02% difference.
lintian should be able to check for misuse of alterna‐
tive and virtual dependencies in Build‐Depends and Build‐
Conflicts.
10. Work items
· Aptitude resolver must terminate build on install‐deps
failure. Not a serious bug, but it means the cause of
failure is reported correctly.
· Should the internal resolver delegate virtual depen‐
dency resolution to apt‐get (current release behaviour)
or revert to aborting all builds using virtual depen‐
dencies?
· apt and aptitude resolvers don’t print the parsed pack‐
age build dependencies, so it’s not possible to
directly check the installed package set matches what
was requested. Print the list as for internal.
--
.''`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20110221/020d69d1/attachment-0001.pgp>
More information about the Buildd-tools-devel
mailing list