[buildd-tools-devel] Bug#865541: Bug#865541: sbuild --apt-distupgrade should not remove build-essential

Mon Jun 26 11:26:20 UTC 2017

Hi,

Quoting Raphael Hertzog (2017-06-26 11:58:04)
> On Mon, 26 Jun 2017, Johannes Schauer wrote:
> > > Then it would make sense to not have APT_DISTUPGRADE=1 by default. Having
> > > sbuild refusing to build in that situation is not really helpful either.
> > Just to make sure I understand you: changing the default would probably not
> > fix your situation either?
> It would because actually I pass "--apt-update --apt-upgrade" but not
> "--apt-distupgrade" and I have no ~/.sbuildrc.

then why not pass --no-apt-distupgrade?

> But then it would likely have to do something else to update the build
> choot. That said I don't think that changing the default is the right
> answer here.

Yes, you would have to distupgrade the buildchroot outside of sbuild using
"sbuild-update" for example or using an external command hook like
--chroot-setup-commands.

> If I call sbuild-update --dist-upgrade, I might understand that you would
> like the command to fail.

Unfortunately, currently sbuild makes no distinction between how a
configuration variable was set, whether it was via the sbuildrc, a commandline
switch, an environment variable or whether the user changed nothing and it was
just the default.

Adding provisions that would let sbuild know who set a variable and then
letting sbuild behave differently depending on how the variable was set would
be a big effort.

> > > Indeed. I don't see why a warning would not be sufficient. Or doing it in
> > > two steps "apt-get install build-essential, apt-mark hold
> > > build-essential, apt-get dist-upgrade"
> > But if you put a hold on build-essential, then the distupgrade will not be
> > able to install the packages it needs to succeed.
> I'm afraid we don't have the same definition of "succeed". It's normal for a
> dist-upgrade to keep some packages on hold when doing otherwise would remove
> packages that the user wants to keep.
> 
> For end users, this happens all the time in unstable during large
> transitions. You don't want to get the latest perl until all the perl
> modules that you have installed have been rebuilt.
> 
> For a build chroot, we want the opposite... as long as it doesn't break
> the build chroot. If the new perl causes the removal of dpkg-dev, then
> it should not be dist-upgraded. If it causes the removal of
> liblocale-gettext-perl only, then it's fine.

Unfortunately, "apt-mark hold" has the nasty side effect that it not only stops
a package from being removed but it also stops a package from being upgraded.
So if a distupgrade were necessary to upgrade build-essential then we wouldn't
want it to be on hold or otherwise build-essential would not get upgraded. So I
don't think "apt-mark hold" is the right thing to do for sbuild in general.
(but it might be the right thing for you to do in which case you could use
--chroot-setup-commands)

> > Having "apt-get distupgrade" either silently (with just a warning but
> > without killing sbuild) fail or not do the full upgrade seems wrong to me
> > because if the user requested a distupgrade and cannot get one, then sbuild
> > should abort.
> Then make the --apt-distupgrade a tri-state option:
> never/always/when-possible and make it default to "when-possible".
> 
> But to me it seems like useless complexity when 99,9% of the users just
> want the build to succeed with the freshest working build environment that
> we can get in the current situation.

Do they? At least I would count myself to the other 0.1% but it's hard to find
out.

But making --apt-distupgrade a tristate option where in addition to
--no-apt-distupgrade we could have --maybe-apt-distupgrade would probably
indeed fix your issue.

> > Otherwise that would be like the user requesting to do a archall-only build
> > but if it cannot sbuild will just issue a warning instead of failing.
> 
> I don't understand the analogy. The archall-only build is really about
> what we want, if we can't get what we want, it should fail. The
> --apt-distupgrade and not about what we want, it's about the environement,
> it's more in the realm of the "how".
> 
> And while you interpret the option literaly as "the user wants a dist-upgrade"
> I believe that it should be understood as "I want a package build in the
> latest build environement available".

As explained above, with the way sbuild currently works there is no way for it
to know where the "I want a distupgrade" comes from. Whether it's from the
command line, the sbuildrc or whether it just picked the default. So with
things are as they are right now, having an option set can only be interpreted
as "the user wants it" independent of whether the user really manually set
something on the command line, made an entry in the sbuildrc or whether the
default was used and the user just "implicitly" chose a certain option.

> > If you are really set on letting sbuild silently fail if "apt-get upgrade"
> > fails you could instruct your buildds to not do the distupgrade by default
> > but you do it manually using --chroot-setup-commands. Using that you can
> > then handle the situation however you like.
> I have plenty of ways to hack a solution but if I file a bug report, it's
> because I believe that it is a shortcoming that should be fixed for all, not
> only for my specific case.

That's very much appreciated, thank you! I feel the same, which is why I try to
find a good solution together with you. Sorry if my arguing comes over as very
dismissive - it is not meant to be. I think much of my irritation just came
from me not exactly having understood your problem. But I think we are making
progress there. :)

> > And does the problem not ultimately come from how kali imports packages
> > from testing? Should you not rebuild all reverse dependencies of packages
> > that you fork to make sure that the source packages you offer have been
> > built against the right binary packages? So would the problem not also be
> > solved if you would also rebuild src:glibc?
> Such additional work would solve this issue it would also multiply the amount
> of work by ten (i.e. much more than having to deal with this specific hiccup)
> and it would offer no benefits for the end users.

Makes sense.

Thanks!

cheers, josch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: signature
URL: <http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20170626/8d341413/attachment-0001.sig>