[buildd-tools-devel] Bug#604268: Bug#604268: Bug#604268: Bug#604268: QEMU linux-user support

[Loïc Minier]

On Sat, Jan 22, 2011, Roger Leigh wrote:
> OK, I've worked on this a bit today, and I've attached a patch for
> the current work in progress.  Basically, I've added support for
> machine emulation as a "chroot facet" inside the actual C++
> sources, rather than the setup scripts.  This means the qemu
> configuration etc. can be done directly in the chroot configuration
> in /etc/schroot.  A patch is attached, as is the documentation
> (searching for "machine emulation" will find the relevant bits).

 Sounds good!

 I think I'm happier with more things being outside of /etc; I am a bit
 unconfortable with the amount of important logic which one is able to
 modify under /etc/schroot  ;-)

> Your 11qemu script has been added as 50emulation (does it need to
> be sooner?)  I renamed it to allow other emulation programs to be
> used potentially; the same applies to the option names.

 I might have forgotten the exact reason I picked 11qemu, but it should
 relatively early as to allow subsequent scripts to run commands in the
 chroot; I seem to recall that some setup.d script where chrooting +
 running stuff.  I think I placed it as soon after the chroot becomes
 usable to chroot into (after the filesystems are mounted).

> I'm not sure if emulation-arch is that useful; is there any reason
> not to always autodetect?  Maybe if autodetection fails?

 One thing I pondered recently is if it would be best to ask the host
 environment's binfmt-support database about the interpreter to copy,
 based on an executable from the chroot.  IOW, match
 /var/lib/binfmts/qemu-* against some executable from the chroot, and
 then find out the qemu binary to use from that file.  Or perhaps binfmt
 could tell us with a new interface, or we could find out by running a
 static binary and looking at its interpreter:
    CHROOT=/var/lib/schroot/mount/natty-armel-f3e92a7f-f1a5-472e-8c0b-3e2c6f8f70cf/chroot-autobuild
    $CHROOT/lib/ld-linux.* --library-path $CHROOT/lib $CHROOT/bin/cat /proc/self/cmdline

 this outputs:
    /usr/bin/qemu-arm-static\0<chroot>/lib/ld-linux.so.3\0 ...
 (the above form might not be terribly portable, but note how we avoid
 the whole dance of running "file", or the name of qemu architectures;
 heck, we don't even need to know it's qemu!!)

>                            Maybe if autodetection fails?  Also, we
> can use emulation=qemu to force qemu usage--since we enable it in the
> configuration explicitly, do we then need to check the host arch to see
> if we should run qemu?  This would make the setup script simpler and
> behave the same under most conditions.

 It's fine to always run qemu even if not needed; just a waste -- I'm
 not sure it's actually honored though, the kernel's matching rule might
 pick up 32-bits binaries before binfmt binaries.

>                                       (I think you mentioned supporting
> other executables?)

 maybe, I don't recall; it would sound wrong to hardcode qemu or the
 path to qemu in any case

> If you're unhappy with any of the names used, that's also trivial to
> change if you like.  (I'm fairly rubbish at naming things!)

 That's probably as good as what I would think of; qemu syscall
 emulation is usually named "CPU transparency" because it's basically a
 mapping of a flow of CPU instruction to another one, with syscall
 translation.  This is different from qemu machine emulation which
 emulates hardware; this is sometimes called simulation.  Upstream, the
 syscall emulation stuff is called "qemu-user emulation".

> BTW, does kfreebsd have the same binfmt-misc and qemu support as Linux?
> I'm just wondering if this should be enabled by default on BSD as well.

 I have no idea; in the past, I thought about a couple of ways qemu
 could be modified to provide the same functionality, but without kernel
 support.  I could think of at least two ways, but didn't pursue any of
 them in the end as binfmt is too convenient; one way would be via
 LD_PRELOAD, and the other way would be to teach qemu that when it
 translates exec() and similar it should actually prepend itself.  I
 think the -L option is also supposedly useful, but broken since ages.

> --- a/configure.ac
> +++ b/configure.ac
> @@ -246,6 +246,18 @@ AC_ARG_ENABLE([union], [AS_HELP_STRING([--enable-union], [Enable support for uni
>                [enable_union="auto"])
>  AC_MSG_RESULT([$enable_union])
>  
> +AC_MSG_CHECKING([whether to enable support for machine emulation])
> +AC_ARG_ENABLE([emulation], [AS_HELP_STRING([--enable-emulation], [Enable suppoer for machine emulation (qemu)])],

 "support" not "suppoer"

> --- a/debian/control
> +++ b/debian/control
> @@ -95,7 +95,7 @@ Architecture: any
>  Depends: ${shlibs:Depends}, ${misc:Depends}, schroot-common (=${source:Version})
>  Conflicts: dchroot (<< 0.13)
>  Enhances: sbuild
> -Suggests: debootstrap, lvm2, btrfs-tools, aufs-modules | unionfs-modules
> +Suggests: debootstrap, lvm2, btrfs-tools, aufs-modules | unionfs-modules, file, qemu-user-static | qemu-kvm-extras

 Ah it's actually qemu-kvm-extras-static in Ubuntu, not qemu-kvm-extras;
 I can see I'm at the origin of this bug

> +# Depends: file, qemu-user-static | qemu-kvm-extras

 Should be | qemu-kvm-extras-static

> +# - allow setting the qemu interpreter in the script-config file

 The idea was allowing to set a qemu interpreter of one's choice,
 without relying on autodetection or to override it; I'm not sure it's
 very important

> +# - allow forcing use of qemu even when not particularly needed

 a bit of the same

> +# - add more biarch cases

 would be nice if we had a way to test for these cases instead of
 hardcoding a list; in fact, I wish I had done so instead of using file.
 (Matching chroot binaries against /var/lib/binfmts/qemu-* to find the
 interpreter.)

> --- a/man/schroot.conf.5.in
> +++ b/man/schroot.conf.5.in
> +If configured to do so, schroot will allow the use of a machine emulator such
> +as
> +.BR qemu (1)
> +inside the chroot environment.  This may be used to build and use software
> +using an entirely different CPU architecture, for example to build programs for
> +the ARM CPU on an amd64 system.

 You might want to stick to GNU names there, e.g. "for the ARM CPU on an
 x86-64 system"

> +Currently, qemu is the sole method for supporting machine emulation.  If one
> +wishes for example to emulate an ARM machine, schroot will copy the static
> +\fIqemu-arm-static\fP binary from the host system to the chroot in order to
> +allow the emulation to work transparently.

 This reminds me that there's an inherent bug in the current approach:
 if you install qemu-user-static in the chroot, you're screwed
 (the x86 qemu-arm-static which you copied into the chroot gets replaced
 by an ARM one).  A quick and dirty way around this would be
 dpkg-divert (run in the chroot just after copying qemu-arm-static), a
 long-term elegant way might be multiarch.


 I've read through the diff, it seemed fine; thanks for the efforts
 you've put into this C++ version!  I would suggest that you proceed
 with this version and we keep the other ideas from above for a second
 version; there is no need to delay this (IMO quite useful) support for
 the new issues I bring up above.

-- 
Loïc Minier