[buildd-tools-devel] testers wanted: sbuild and build-dependencies

[Goswin von Brederlow]

Roger Leigh <rleigh at codelibre.net> writes:

> On Wed, Nov 10, 2010 at 01:00:29PM +0100, Goswin von Brederlow wrote:
>> Roger Leigh <rleigh at debian.org> writes:
>> 
>> > A new "aptitude" resolver has been written by Marc 'HE' Brockschmidt.
>> > This creates a dummy "dependency package" which is installed and
>> > contains Depends and Conflicts for all the appropriate Build-Depends
>> > and Build-Conflicts, and uses aptitude to install and remove the
>> > appropriate packages to satisfy them.  This has been in use on our
>> > experimental buildds for about a year, and we now want to look towards
>> > making it the default.  However, we need some more widespread testing
>> > to make sure the dependency resolving doesn't result in inconsistent
>> > installation of packages and hence inconsistent library dependencies
>> > or break building of any packages etc.
>> 
>> Yet another or did he take the one from pbuilder?
>
> It's new, but based upon the ideas in pbuilder.
>
> Just to mention in passing, I also wrote an additional "apt"
> resolver last night, based on the aptitude resolver, which works
> in exactly the same way.  Not usable under all circumstances yet,
> but also available (in git only at present) for testing.  The
> main issue with apt is getting "apt-get -yf install" to not choose
> to remove the dependency package as a solution!  Currently looking
> at the possibilities here--any thoughts welcome!

Does that actually happen in cases where another solution would keep it?

There is a solution to this actually. Create a Packages, Release and
Release.gpg file for the pseudo package and add them as file:// url to
sources.list.d/. Then just apt-get install pseudo-package.

>> This destroys the determinicity of build dependencies. So if I say
>> 
>>   Build-Depends: lib-new-name | lib-old-name
>> 
>> so the package builds (for users) in both stable/testing and unstable it
>> is no longer given which library is used by the unstable buildds. Some
>> will pick lib-new-name and some, where the new lib isn't compiled yet,
>> lib-old-name. And so on.
>> 
>> I always heard determinicity was a wanted feature for the buildds.
>
> It is, and this is something to look at carefully.  In the example
> above, this is a very real problem (which could be rectified by
> having stricter build-deps).
>
> The root problem here is that at any given moment in time, unstable
> is not in a consistent state--packages may be outdated on several
> architectures, and so a package build may use (and depend on)
> different versions in different architectures.  There are several
> places this could be fixed:
>
> • we could keep packages out of the archive until built on all
>   architectures.
> • we could dep-wait a package until all its deps are the same
>   across all architectures

That is called testing. And for anything doing a library transition that
obviously won't work or migration to testing wouldn't be such big
problem all around.

Don't forget this not only happens when a package itself differs between
arch. This can also happen if any of the recursive depends differs. On
one arch lib-new-name can exist but be uninstallable while on another it
is installable.

> • we could keep the current approach of banning alternatives
>   entirely (but note that this does not solve the entire problem,
>   it's still easy to get inconsistency anyway)
>
> The existing approach to determinism is not to support alternatives
> at all, which is clearly not ideal.  While I don't think we should
> be encouraging the use of alternative build-deps, this is one of the
> most commonly reported bugs in sbuild--there are valid use cases for
> them.

Actually alternatives are supported. Most notably in A [i386] | B
[!i386]. Sbuild fixates on the first alternative that should be
installable. That makes it 100% perdictable to the uploader which
package he gets.

> All the modern buildds are based around snapshotting the build chroot,
> which gives us a clean slate at the start of each build.  Providing we
> have a common base environment across all our buildds, this should
> provide a measure of stability--I don't think any of the existing
> resolvers choose packages randomly--so the resolver should pick the
> same package sets across architectures so long as the available
> packages are the same.  Now, this might not always be true, but this
> also affects the existing resolver as well.  Not letting binary
> packages enter unstable until built on all architectures would be a
> solution to this.  I'd have to say, solving the problem at its root,
> and giving our tools more flexibility would be the ideal outcome.

Problem is packages aren't the same and enforcing sameness causes too
much delay and deadlocks. Just look at testing.

MfG
        Goswin