[Pkg-mc-devel] Bug#805495: mc: Very slow opening of an ISO file

[Thomas Schmitt]

Hi,

in friendly competition to Brian's patch i am now exploring
the possibility of a broker script which implements much of
my proposed mc roadmap.

A demo script for /bin/sh is now uploaded to SVN
  http://libburnia-project.org/browser/libisoburn/trunk/frontend/xorriso_broker.sh?format=txt
(203 lines, 4.7 KB)
It needs xorriso-1.3.2 or newer. Current upstream release is
  https://www.gnu.org/software/xorriso/xorriso-1.4.0.tar.gz

It expects an id string, by which it finds the named pipes
of the running xorriso process. If no such pipe set is found,
pipes are created and xorriso gets started.

Although it is shell code it is quite fast as soon as the ISO
meta data is loaded by the first mc operation in the ISO image.
On a small ISO it does 100 -ls commands in 0.3 seconds
compared to 1.8 seconds needed for 100 xorriso runs with
-dev and -ls.
With DVD+RW in /dev/sr0, the penalty for -dev is massive.
Ten times the broker script beats ten individual xorriso
runs by 7 to 60. (90 more broker runs last 0.25 seconds.
90 more xorriso runs would nearly last 10 minutes.)

Still missing are solutions for the following mc problems:

- How to get a unique id for the mc instance ?
  Does
    echo "PPID=$PPID" >&2
  in the iso9660 script repeatedly report the same number ?
  In this case
    ID="mc_$PPID"
  would be suitable.

  Lame workaround:
  Have one common xorriso slave for all mc instances on the
  machine. This might cause unnecessary -commit of sessions
  if changes are pending while the other mc issues its own
  xorriso commands. In any case it will cause unnecessary
  loading of ISOs if more than one is in use.
    ID="mc"

- How to let the user express the wish to commit a set of
  change operations to the ISO image file as new session ?

  Lame workaround:
  Have a dummy ISO image file  commit.iso  and let mc
  show its (nearly empty) content. This switch of ISO filesystems
  will cause pending changes of the previous -dev to be committed.

- How to cause mc to send xorriso command "-end" or
  "-rollback_end" immediately before mc itself ends ?

  Lame workaround:
  Send -end via the broker script, using the id of deceased mc.

-----------------------------------------------------------------

Only with this architecture, the write capabilities of xorriso
can be used properly. The current way in iso9660 is very awkward.

See a demo:
(Files /tmp/test.iso and /tmp/commit.iso do not yet exist)

  XORRISO=...path.../xorriso_broker.sh
  ID=demo
  ISO=/tmp/test.iso

  "$XORRISO" $ID -dev "$ISO" -lsl / --

shows an empty (actually not yet existing) root directory.
Let's put some files into it

  "$XORRISO" $ID -dev "$ISO" -map /usr/bin /ubin

They get shown already. But still no /tmp/test.iso exists:

  "$XORRISO" $ID -dev "$ISO" -lsl /ubin -- | less

We could now -map more files, -rm some, -mv some, ...

Let's force writing of the initial session:

  "$XORRISO" $ID -dev /tmp/commit.iso -lsl / --

Now /tmp/test.iso gets written and then empty /tmp/commit.iso
says "total 0".
(More straightforward would have been
  "$XORRISO" $ID -dev "$ISO" -commit
 but there is no suitable method of mc to see in iso9660,
 which could trigger this command.
)

Let's switch back to the written ISO and weight it:

  "$XORRISO" $ID -dev "$ISO" -du / --

(Because no changes were made to commit.iso, it does not get
 written now.)
  
End the xorriso process which was running all the time

  "$XORRISO" $ID -dev "$ISO" -end

At the next invocation with $ID a new one will be started
and load the meta data from "$ISO".

Clean up

  rm "$ISO"

If "$ISO" is a DVD drive (e.g. /dev/sr0) then rather end by

  "$XORRISO" $ID -dev "$ISO" -eject all -end

(and do not remove /dev/sr4)

The shown commands would also work the old mc style if
  XORRISO=xorriso
  ID=""

-----------------------------------------------------------------
Necessary changes in mc's iso9660 script:

Replace this line by code which finds the broker script
(e.g. as neighbor of the iso9660 script):

  -XORRISO=$(which xorriso 2>/dev/null)
  +XORRISO=$(whatever is needed)

-------------

Determine the id string of the calling mc instance.
It must stay the same for all invocations from that mc
instance. Hopefully:

  ID="mc_$PPID"

-------------

Change all occurences of $XORRISO to

  "$XORRISO" $ID

-------------

Take care that the -dev command is always the first of
the arguments which follow the id string:

  -   lsl=$( $XORRISO -abort_on FATAL -dev stdio:"$1" -cd "$dir" -lsl 2> /dev/null )
  +   lsl=$( "$XORRISO" $ID -dev stdio:"$1" -cd "$dir" -lsl 2> /dev/null )

(The command -abort_on FATAL is counter productive anyway.
 The slave process gets started with -abort_on NEVER.)

-------------

The file names handed over to xorriso should be enclosed in
quotation marks with proper escaping. This sh function does
what is necessary:

  # Make filenames safe for transport by wrapping them in quotes and
  # escaping quotes in their text
  xorriso_esc() {
    echo -n "'"
    echo -n "$1" | sed -e "s/'/'"'"'"'"'"'"'/g"
    echo -n "'"
  }

To be used e.g. as

  -    $XORRISO -dev stdio:"$1" -osirrox on -extract "$2" "$3" >/dev/null 2>&1

  +    "$XORRISO" $ID -dev stdio:"$1" -osirrox on \
  +       -extract "$(xorriso_esc "$2")" "$(xorriso_esc "$3")" >/dev/null 2>&1

-------------

Optionally remove all "stdio:" prefixes.
They prevent operation on optical media.
They are needed only if you want to address other device files
like /dev/null or /dev/sdc. Those are protected against mistyping
by the demand for a "stdio:" prefix.
(The superuser doing a backup will be happy to be kept from
 overwriting the system disk.)

-------------

Have a nice day :)

Thomas