[pkg-fso-maint] d-i installation on embedded devices (was Re: MTD Installation)

[Colin Watson]

On Thu, Mar 26, 2009 at 04:47:28PM +0000, Per Andersson wrote:
> My first question is if it is possible to run d-i without having to
> boot it. Use case would be to use the nandsim module (maybe from
> inside a chroot) "live" on my normal installation instead of having to
> boot the installer in, say, QEMU.

There are some targets for this in d-i's build system, but they'll only
work for some things since d-i rather expects to control the system from
the ground up; I'm not sure I'd be brave enough to try partitioning
using them.

> Second question: If that is not possible, how do I emulate MTD with
> QEMU. AFAIK it is not possible with -M versatilepb, which Debian
> provides kernels for. This would mean that I would need a custom
> kernel, how does that work together with d-i?

Look up kernel-wedge; there's documentation on custom kernel handling in
the d-i wiki too.

> Third question: What do you think about the scope of the MTD
> Partitioning and Installation project? Would it fill 3 months? (I
> think it would.)
> 
> So far I propose creating partman modules:
> * partman-ubi
> * partman-auto-ubi
> * partman-ubifs
> 
> Of course also documentation of how everything
> works.
> 
> Suggestions on scope and/or additional tasks
> (that I might have missed)?
> 
> As an exercise I give a draft of an implementation
> proposal below. Please comment wildely.
> 
> For the modules scripts would be built for init.d,
> display.d, check.d, commit.d, finish.d etc.
> 
> Example partman-ubi:
> * init.d -> script that recognizes MTD device(s) and
>   loads appropriate kernel modules.
> * display.d -> partitioning the MTD, create data
>   structures based on user input. (Exit code set
>   properly, 100 for success, 255 for failure and 0 for
>   no action performed.)

display.d is essentially for automatic partitioning, and would belong in
partman-auto-ubi rather than partman-ubi. In any case, in order for this
to be useful you'll need to define a recipe format. I wouldn't start
with this. Start with getting manual partitioning to work first, and to
write the right structures into /var/lib/partman/devices/. In order to
do that you'll need to define what those structures should be. Ideally,
they should follow normal partitioning as closely as possible, but I
don't know whether that will be possible.

As I think I've mentioned before in the context of this proposal, I
would suggest a prior period of investigation into how partman-lvm (and
partman-auto-lvm) works. The reason I suggest this is that my
understanding of UBI is that it's essentially logical volumes for MTD,
and this suggests that you ought to learn from partman-lvm while
avoiding the same mistakes (I'm not suggesting that you copy it). You
should be conscious of the flaws of partman-lvm, for example:

 * partman does not know accurately in advance what size LVM objects are
   going to be, and so it has a two-phase approach: first, you use a
   separate LVM configuration tool (accessible from the partman menu) to
   set up LVM objects, and then partman-lvm commits this to disk before
   allowing you to operate on the logical volumes. You should understand
   the details of this and how it might be avoided, along with building
   a good mental model of how partman's commit phase works, before
   proceeding.

 * The partman-auto recipe format does not allow for nesting, so our
   approach to tasks such as "create a logical volume in this volume
   group containing the following filesystem" is at best baroque
   (although it is at least possible to do this as of relatively
   recently). It gets worse when RAID is involved.

 * partman thinks of logical volumes as more like disk devices than
   partitions, but disk devices that can only hold one partition. This
   results in a somewhat strange UI in some respects, and it would be
   worth thinking about this.

> * finish.d -> write partition scheme to fstab etc.

Let partman-target do this; partman-ubi probably shouldn't do anything
with fstab at all. partman-ubifs would need to have a fstab.d script.
All you should need to do at the partman-ubi level is ensure that the
proper device information is written to /var/lib/partman/devices/, as
other block device providers do.

I expect that you'll also need to come up with patches to quite a number
of other bits of d-i to handle device name changes.

> As a ball park figure I expect that this would 3 weeks
> to finish. Opinions?

I think that's probably about right for the partman-*ubi* core packages,
but anything that involves support for new devices names tends to
involve scattergunning patches across lots of d-i, and you will need to
think about boot loader support. Make sure to add time for investigation
and familiarising yourself with d-i code, which is not trivial in some
places.

> Lastly, is there someone willing to mentor this?

I can certainly help with code review, but mentorship of this is
probably a bit much for me; I strongly feel that it should be somebody
with some familiarity with MTD and UBI. I would be willing to help such
a mentor out informally.

-- 
Colin Watson                                       [cjwatson at debian.org]