Packaging Zope 2.12

Tue Jul 27 13:20:53 UTC 2010

Hey Michael,

On 26/07/2010 Michael Mulich wrote:
> On 7/20/10 6:14 PM, Jonas Meurer wrote:
> >On 14/07/2010 Michael Mulich wrote:
> >I for sure can help you with packaging issues, but unfortunately I don't
> >know much about buildout and all that distutils stuff. Maybe we can work
> >together in order to package zope2.12, that would be awesome.
>
> Sounds like a plan.

Indeed. Let's try it ...

> >Do you know the pkg-zope SVN repository? It already has a zope2.12
> >directory, but I guess that most of debian/rules (the makefile used to
> >build the package) needs to be rewritten. You can find the repos at
> >svn://svn.debian.org/svn/pkg-zope/zope2.12
> >
> Thanks for the link. I found the repository the other day. I decided
> to fork the zope2.12 package for now, but would be more than willing
> to work on the package in the debian repo if someone can give me
> access.

Sure, but first you need an account on alioth. if you don't have one
yet, you can register at http://alioth.debian.org/

> >If I got it right, the main difference between zope2.11 and zope2.12
> >build system is, that zope2.12 no longer contains the required zope
> >dependencies (libraries developed for zope). Instead it uses some
> >buildout receipe to fetch these dependencies as python eggs, and builds
> >something like a full python environment, including all required python
> >dependencies.
> >
> Right. My latest stab has been to create a buildout recipe that will
> isolate these dependencies.[0] I've tried to solve the dependency
> issue without using buildout, but I'm tired of fighting that battle,
> so I've embraced the buildout. I intend to use this isolation recipe
> to populate a python environment (or virtual environment) rather
> than trying to build Zope inside the that environment.
> 
> The isolation recipe has three deliverables that are relevant to the
> packaging of zope:
> 
>    1) The distribution packages and any dependencies are placed in a
> directory.
>    2) A .pth file, which is basically a list of paths where these
> packages are located, is created; and placed by default along side
> the packages.
>    3) The scripts that get generated from these distributions are
> placed into a separate directory.
> 
> I created a document that somewhat illustrates the new build process
> for the zope2.12 package.[1] Rather than repeat all that here, I'll
> just supply the link at the end of this email under number [1]. But
> for completeness, the gist of it is that we can use the .pth file to
> pull all the zope dependencies into a single python environment.

I took a quick look at it, see below.

> >So one issue with packaging is how to make the dependencies available to
> >the build process. Fetching them from internet is not an option at all.
> >Unfortunately the dependencies break even for minor version changes (in
> >other words: they don't ensure backwards compability), thus packaging
> >all the zope libs as seperate packages is not an option either. Thus the
> >only option is to distribute all required eggs within the zope package
> >tarball, which is a bad solution security-wise.
>
> Can you elaborate on why that would be "bad solution security-wise".

because some of the dependencies are already packaged as seperate
packages within debian. these packages are updated regularely for new
upstream versions. as zope2.12 requires particular version of these
dependencies in order to build, using the debian packages as build
dependencies is no option. on the other hand, shipping older versions of
the dependencies within the zope2.12 package creates a security problem:

code is duplicated. both the zope2.12 debian package and the zope
modules packages do contain the same software. if a security issue
requires a package update by the security team, the security team needs
to know that zope2.12 contains a copy of this code.

> >Another issue is how to build the zope/python environment in a FHS
> >compliant way: placing the different parts of zope (libraries, logs,
> >configs) at places similar to legacy zope versions: with relative paths,
> >using the system python interpreter instead of a local copy, etc.
> >
> This is the part I'm still fuzzy on. I'm not very familiar with the
> legacy zope setup. I currently use the zope2.10 package, and I'm
> only somewhat somewhat familiar with that.

you can see the structure of zope2.10 package with 'dpkg -L zope2.10'.

basicly this is:

/usr/lib/zope2.10:
	/bin:		the zope binaries such as repozo.py
	/skel:		the skeleton files for new zope instances
	/lib/python:	all shipped python and zope modules

/usr/share/doc/zope2.10: documentation stuff

/usr/lib/zope2.11/bin/python: a symlink to the system python version

and some empty directories which are used by instances:

/var/log/zope2.10/<i_name>:		<i_name> log files
/etc/zope2.10/<i_name>:			<i_name> config files
/var/lib/zope2.10/instance/<i_name>:	all other <i_name> files

/usr/share/zope/Products: Zope Products installed as Debian packages

this structure should stay as is if possible. 

> >And last but not least instances need to be seperated from the global
> >zope2.12 files in a similar way as done up to zope2.11. continuing
> >support for the debian 'dzhandle' utility for managing zope instances,
> >products and extensions should be the goal.
> >
> I'll have to think about this a bit more. I decided early on in this
> process that I was going to abandon Products support, because 1)
> Plone has decided to remove the products section from it's default
> buildout 2) the unified installer has done this as well. Therefore,
> I think it would be easier for us to drop the Products stuff. But I
> would like to add support for adding distributes (aka eggs) much
> like products are added currently.

I'd like to keep the Products support, and if I got it right that will
not add any extra work for packaging. Zope2.12 already contains support
for Products, Debian tools (dzhandle) just needs to manage symlinks from
the instance's Products folder to actual place of Products.

> >greetings,
> >  jonas
> Thanks for the support Jonas!
> 
> I'd be grateful if a few people could look over the document [1] and
> let me know if there are any glaring misunderstandings and/or
> missing pieces. Thanks.
> 
> [0] https://weblion.psu.edu/trac/weblion/browser/weblion/buildout.recipe.isolation?rev=12091
> [1] https://weblion.psu.edu/trac/weblion/browser/users/mrm41/zope2.12-debian_packaging_notes.rst?rev=12092

this looks very promising. but am I right, that you want to install the
python environment to /usr/lib/zope2.12 at the build process? that is
not possible. The whole build process needs to be done inside the
package directory. In most cases, Debian packages build the package into
./debian/<pkg-name>/. If zope2.11 is built from /home/user/zope2.11/ as
example, the package will be compiled and built into
/home/usr/zope2.11/debian/zope2.11/, and this directory contains the
whole filesystem structure of the package:
/home/usr/zope2.11/debian/zope2.11/usr/lib/zope2.11 ...
/home/usr/zope2.11/debian/zope2.11/usr/share/doc/zope2.11 ...
and so on.

thus the build process needs to take care of paths in the built package.
this is done by configure/build flags for software built with autotools.
Unfortunately, I don't know how to do this with buildout.

If you have any further questions, don't hesitate to ask.

greetings,
 jonas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-zope-developers/attachments/20100727/ebf5ce37/attachment.pgp>