[buildd-tools-devel] Bug#809730: [Multiarch-devel] multiarch cross build dependency resolution

Wed Jan 6 11:35:10 UTC 2016

Hi,

Quoting Johannes Schauer (2016-01-06 11:19:18)
> Quoting Roger Leigh (2016-01-06 10:51:53)
> > The major difference in behaviour is the handling of alternative build 
> > deps.  Unless something has changed, apt-get build-dep didn't provide 
> > the deterministic behaviour of sbuild's resolver.
> 
> [...]
> 
> But you bring up a good point! Even when apt 1.1 hits old-old-stable, we could
> still not switch to a "apt-get build-dep foo.dsc" based system if we want to
> retain sbuild's ability to mangle build depends:
> 
>  - to remove alternatives
> 
>  - to inject additional dependencies via --add-depends and friends
> 
> I'll update my commit message with that note.

my reworked reasoning for how sbuild will do things (and my adapted commit
message) is the following.  Please correct me where I'm wrong:

  | Three decisions are to be considered:
  | 
  |  a) does sbuild need to use an internal dummy repository?
  | 
  |  b) will sbuild use 'apt-get install' or 'apt-get build-dep' or both?
  | 
  |  c) can advanced apt features like 'apt-get build-dep foo.dsc' not be
  |     used?
  | 
  | Maintaining an internal dummy repository instead of offloading all the
  | complexity to apt results in complex code required to reside in sbuild
  | itself. The justification is as follows:
  | 
  | Moving complexity to apt makes sbuild depend on apt *inside* the chroot.
  | This can have negative effects if building packages in a very old chroot
  | (like old-old-stable). Mangling build dependencies allows us to take
  | care of new features introduced in the archive (like the :native
  | qualifier or build profiles) while the apt inside the chroot does not
  | need to understand them. This means that sbuild is able to build for
  | example a build profile enabled binary package on Squeeze or Wheezy
  | because a newer libdpkg-perl from the host system takes care of mangling
  | the build dependencies correctly. This is for example useful for
  | backports. Apt 1.1 has the feature of being able to call "apt-get
  | build-dep foo.dsc" but this feature cannot be used by sbuild until that
  | apt version is in old-old-stable or otherwise sbuild would not be able
  | to use old chroots.  We cannot control the apt that is used inside the
  | chroot but we can control sbuild by doing new uploads to unstable. So by
  | using sbuild to do the heavy lifting we can make sure that new
  | dependency features (like :native and build profiles) can used even with
  | chroots with an apt version that doesn't support them.
  | 
  | Package dependency mangling is required. It is necessary to filter build
  | dependency alternatives as well as support for injecting additional
  | build dependencies via --add-depends and other command line options. So
  | the complexity of using Dpkg::Dep::deps_parse to parse the build
  | dependencies cannot go away. A nice side effect is that as explained
  | above we can take care of newly introduced dependency features so that
  | apt inside the chroot doesn't have to.  One disadvantage of maintaining
  | an internal dummy repository used to be that it requires keys to be
  | generated (which would be too slow on some virtualized machines or
  | containers) but this has been solved with commit aea9a4d9 or closing of
  | bug #801798 using apt-get's feature of [trusted=yes] in sources.list,
  | making key generation optional and only required for chroots of squeeze
  | or older.
  | 
  | Building dummy binary packages has been tested for a long time and seems
  | to be a reliable way to install build dependencies. Unless there is a
  | shortcoming of this method, there is no reason to switch to "apt-get
  | build-dep" to install dependencies from a mangled dummy Sources file.
  | If the "apt-get build-dep" method was to be introduced it would have had
  | to be tested with all apt versions down to old-old-stable. The risk of
  | breaking things is too great for little gain. The only gain would be the
  | decreased complexity from not needing to build a dummy binary package.
  | But I do not think this gain is worth the risk of breaking working
  | setups.
  | 
  | Maintaining both code paths "apt-get install" as well as "apt-get
  | build-dep" would be too complex because we need to correctly mangle
  | build dependencies for both. Thus we unify build dependency installation
  | using only one method. As explained above we are using the "apt-get
  | install" method to install a dummy binary package because this method
  | has been tried and tested for a long time.
  | 
  | In this commit we switch the method of installing cross build
  | dependencies from using "apt-get build-dep" to installing a dummy binary
  | package of the host architecture as it is used for the native case
  | already. Using "apt-get build-dep" to install cross build dependencies
  | should not be necessary because cross build dependency resolution should
  | not be different from cross architecture binary dependency resolution as
  | far as multiarch is concerned. Build-Depends specific constructs are
  | taken care of either by Dpkg::Dep::deps_parse (architecture qualifiers,
  | build profile restriction lists) or by rewriting the ':native' qualifier
  | to the correct architecture in course of commit d4c137190 or fixing of
  | bug #777216.
  | 
  | In summary, answering the initial questions:
  | 
  |  a) we need to keep using an internal dummy repository because we need
  | to mangle dependencies.
  | 
  |  b) sbuild will continue creating a dummy binary package instead of
  | creating a Sources file because that has been tried and tested with apt
  | versions down to old-old-stable. Only that method will be maintained
  | because having support for "apt-get install" as well as "apt-get
  | build-dep" would add complexity with no reason.
  | 
  |  c) apt features can only be relied upon once they are part of
  | old-old-stable

Thanks!

cheers, josch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: signature
URL: <http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20160106/8f5b9401/attachment.sig>