[buildd-tools-devel] Bug#610689: sbuild: cross build support

[Roger Leigh]

On Thu, Mar 10, 2011 at 04:52:01PM +0000, Hector Oron wrote:
> Hello,
> 
>   I have a set of patches at:
>     < http://git.debian.org/?p=users/zumbi/sbuild.git;a=shortlog;h=refs/heads/zumbi/cross-build
> >
> 
>   We still do not know how to properly hook cross dependency
> resolvers, but there are a couple of options (embuilddeps and xapt)
> provided by `xapt' package.
> 
>   Basically I am doing the following:

[...]
The first part all looks good.

>   Install cross toolchains in the chroot
>     schroot -u root -c /srv/chroot/sid-amd64-sbuild
>     echo "deb http://emdebian.org/debian squeeze main" >>
> /etc/apt/sources.list.d/emdebian.list
>     apt-get update
>     apt-get install linux-libc-dev-armel-cross libc6-dev-armel-cross \
>         gcc-4.3-arm-linux-gnueabi g++-4.3-arm-linux-gnueabi

This part can be done by the xapt resolver directly, without any
special user setup required.

If you look at the apt/aptitude resolvers, you'll see they add a
local apt source by creating a file in /etc/apt/sources.list.d/
too.  Providing you add a configuration variable for the line so
it's not hardcoded, this would be fine.  It will then update at
the normal update/distupgrade stage, so long as you set up the
apt source in the resolver setup() function (and delete it in the
cleanup()).  You'll then want to force the APT_UPDATE configuration
variable to 1 to make it update.

For the cross-packages, you can add these to the core build
dependencies list, so they will be installed along with build-essential
etc.  Or (better) you can add a separate XAPT_DEPENDS variable which can
be appended to CORE_DEPENDS in Build.pm prior to installing the core
dependencies.  Does xapt not bring in these packages directly, or do
they always need installing by hand?

>   Having made a chroot called sid-amd64-sbuild build a package with
>     sbuild -d sid-amd64-sbuild hello_2.6-1
> 
>   Cross build that same package with
>     sbuild -d sid-amd64-sbuild hello_2.6-1 --host=armel --build=i386
> 
>   Resolve cross dependencies calling
>     sbuild -d sid-amd64-sbuild --host=armel --build=amd64
> --build-dep-resolver=xapt mc_4.7.0.9-1.dsc
> 
>  Issues I am having...
>   1. schroot sessions do not end, so I end up having a bunch of sessions opened

Does this always happen, or just with --build-dep-resolver=xapt?
If you add --verbose to the schroot options ($schroot_options=['--verbose'])
does this tell you anything about why it's not deleting them?
Note: $end_session=0 will deliberately make sbuild not end sessions;
is this set?

>   2. xapt resolver is not doing its job, alternatively embuilddeps
> could be used.
> 
>   Brief explanation on cross resolvers:
>   1. xapt
>      `xapt' should work if a list of packages is passed in its command line:
>        xapt -a $HOST_ARCH $PACKAGE_LIST
>   2. embuilddeps
>      `embuilddeps' parses control file information to extract the
> packages list, then passes this list to xapt
>      Usually `embuilddeps' need to be run under unpacked package
> directory: embuilddeps -a $HOST_ARCH
>      Neil Williams, `embuilddeps' maintainer has updated this package
> to allow -d option, so one can tell directly to embuilddeps where is
> the unpacked package to resolve dependencies.
>      I.e.,
>        `embuilddeps -a $HOST_ARCH -d /path/to/unpacked/directory'

Well, when we install the build dependencies the package isn't yet
unpacked.  The unpacking is done in the build() function.  But there's
no reason we can't reorder it if we need to.  The unpacking could
be split out of build() into a separate unpack_source() function,
which could be run earlier if xapt/embuilddeps is in use.

What information does embuilddeps require from the unpacked source
package?  Is this information also available in the .dsc? (We do
have the .dsc information directly to hand).

xapt is doing two jobs it seems:
1) from the package list, make a list containing all the cross packages
   needed
2) install them (but it doesn't list them in the list of packages to
   install, which makes removal harder potentially)

From the resolver POV, what would be really good if it could just do
(1).  This way, we can just give the expanded list back to the
resolver for it to install.  Because we need to clean up after
ourselves, the apt and aptitude resolvers build a dummy
"dependency package" from the dependency list, and then get apt/
aptitude to install it.  We then reverse the process to clean up
after the build.  So ideally, we don't really want xapt to do any
installation: we just want it to give us the package list.  If
this is possible, it might not even be require to have a separate
XaptResolver--we just expand the cross-dependencies and give them
to the regular resolver.

If xapt can't do this directly, how complex is the code that
implements (1)?  Would it be possible to split it out as a separate
tool?

>   It would be really nice if we enhance sbuild with cross support.

Absolutely.

I've rebased your patches against current sbuild.git master, and
they may be found here: git://git.debian.org/users/rleigh/sbuild.git



Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20110310/1b16706c/attachment.pgp>