[Debburn-devel] schilyutils upload

[Steve McIntyre]

On Wed, Aug 12, 2009 at 06:13:33PM +0200, Thomas Schmitt wrote:
>Hi,
>
>i now read 
>http://atterer.net/jigdo/debian-jigdo-mini-howto/x157.html#THE.TEMPLATEFILE
>
>Steve McIntyre wrote:
>>  * contents of files that we know about (i.e. listed in the -md5-list)
>>   are referenced by checksum in the .template file instead of being
>>   included directly. The md5->filename lookup information is stored
>>   in the .jigdo file.
>
>This would be easily done immediately before
>image generation but independently of it.
>We will have MD5 computation in the next release
>of libisofs, anyway.
>
>I understand -jigdo-map defines a prefix
>for paths which match a certain pattern.
>Are there more than one -jigdo-map option
>allowed ?

Yes, you can have many.

>>  * any other data that does not match known files is compressed and
>>    stored directly in the .template file (e.g. filesystem headers,
>>    directories, padding)
>
>This can be done before image generation if
>it is not important to record exactly the same
>file content as in the image in case of race
>conditions with changing files on disk. 

No, it's very important. If the contents in the jigdo file and what's
in the image don't match then you've got problems. For performance
reasons you also only want to read / checksum / whatever the file data
once if possible.

>What compression algorithm is to be used ?

gzip and bzip2 are currently supported; there's scope to plug more in.

>Is there a description which is a bit more
>detailed than this one in the jigdo howto ?
>.template:   |xxxx| md5-0  |xx| md5-1  |xxx|cccccccc|x| md5-3  |xxxx|

Look in the jigdo source package, doc/TechDetails.txt, for *much* more
information.

>> I hooked into mkisofs/genisoimage by simply
>> targetting all the places that write to the
>> output ISO image,
>
>If it is only about the content bytes
>of regular data files, then this happens
>at 
>  int filesrc_writer_write_data(IsoImageWriter *writer)
>in
>  libisofs/filesrc.c
>
>But in my eyes it is cumbersome to deduce the
>file path, decide about its jigdo fate (data
>in template or not) and to write to the hard
>disk while one is at the same time reading
>from hard disk to generate the image stream
>which goes directly onto media ... (wheeze).

It should be easy to add an extra call to a jigdo helper function at
the right point; that's exactly how I plugged into mkisofs. Looking at
the code in libisofs, I think I can see how to do it easily.

>A main problem is to make the jigdo option
>parameters available to the file content
>writer object. Lots of up-wiring through
>the object hierarchy might be needed.

Meh, just a simple matter of code... :-)

>My idea is to rather traverse the directory
>tree and perform these tasks before image
>generation begins.
>This implies to read the file data twice but
>it would avoid any interference with ISO image
>generation.
>(I confess to have bloated
> filesrc_writer_write_data() recently by adding
> MD5 content checksumming.)
>
>My idea would be implemented as libisofs API
>function which gets the jigdo parameters
>as function parameters. To be run by the
>application immediately before image generation.

I/O is the biggest cost of generating an ISO image; you've just
doubled the amount of I/O needed on the input side if you go this way.

>Ouch.
>We planned to implement UDF for the video
>players ... but HFS ? Can't they read ISO ?
>Are there important use cases ?

Well, if you want to boot a CD on Macs they need HFS information
too. Or so I'm told. I'd be happiest if they just followed standards
here too, but it's not looking likely... :-(

-- 
Steve McIntyre, Cambridge, UK.                                steve at einval.com
"C++ ate my sanity" -- Jon Rabone