Introduction

This recipe adds additional filestorage and zodb_db stanzas to the zope.conf and zeo.conf files generated by the plone.recipe.zope2instance and plone.recipe.[zope2]zeoserver recipes. It also creates the directories in which the extra filestorage data files will be created. This makes it easy to add additional filestorages and mountpoints to a Zope 2 instance.

It may also be used to add additional storages to a standalone Zope instance (zeo-client = false in the Zope part); in this case, location is the only client/server option that applies.

Supported options

parts

A list of filestorage sub-parts to be generated, one per line. (This is different from buildout parts.)

zeo

The name of a plone.recipe.zope2zeoserver or plone.recipe.zeoserver part to which we want to add the extra filestorage. Defaults to the first such part in the buildout, if any.

zopes

A list of names of plone.recipe.zope2instance parts to which we want to add the extra filestorage. Defaults to all plone.recipe.zope2instance parts connected to the associated zeoserver part, if any, or all plone.recipe.zope2instance parts, if no ZEO is found.

backup

The name of a collective.recipe.backup (>= 2.7) part to which we want to add extra filestorage/blobstorage backup entries.

The following options affect the generated zope.conf and zeo.conf. Each may be specified for all filestorage subparts in the collective.recipe.filestorage buildout part, or for one particular filestorage subpart by placing the option in a new buildout part called filestorage_subpart, where subpart is the name of the subpart listed in the parts option of this recipe. The name of the subpart may be interpolated by using %(fs_part_name)s in the option.

Some options only apply to the ZEO server or the ZEO client (Zope instance), while others apply to both.

ZEO server options

location

The location of the data file for this filestorage, relative to the buildout root. May also be specified as an absolute path. Defaults to var/filestorage/%(fs_part_name)s/%(fs_part_name)s.fs

This option is also used for <filestorage> sections in standalone Zope instances, but not by ZEO clients.

Server and client

zeo-storage

Set the id of the ZEO storage. Defaults to %(fs_part_name)s.

blob-storage

Set the directory to be used to store blobs for a standalone Zope instance or a ZEO server. Optional. Required if you're going to store blobs, though. Recommended value: var/blobstorage-%(fs_part_name)s

Zope instance (ZEO client) options

allow-implicit-cross-references

Allow references across mounted databases. This is generally a bad idea. Defaults to False.

zodb-name

The name of the ZODB. Defaults to %(fs_part_name)s.

zodb-cache-size

Set the ZODB cache size, i.e. the number of objects which the ZODB cache will try to hold. Inherits from the associated Zope part. Defaults to 5000.

zodb-mountpoint

Set the path to the mountpoint. Defaults to /%(fs_part_name)s. If the object is being mounted at a different path than its location in the source database, two paths may be provided using the virtualpath:realpath syntax. For example, when mounting /source/bar from source.fs at /foo/bar in target.fs: /foo/bar:/%(fs_part_name)s/bar. Note: the name of the mount point and the mounted object must be the same, i.e. /foo:/bar will fail.

zodb-container-class

Set the class of the object being mounted. Defaults to not being set.

zeo-address

Set the port of the associated ZEO server. Inherits from the associated Zope and ZEO parts. Defaults to 8100.

zeo-client-cache-size

Set the size of the ZEO client cache. Inherits from the associated Zope part. Defaults to '30MB'.

zeo-client-name

Set the name of the ZEO client. Defaults to %(fs_part_name)s_zeostorage.

zeo-client-client

Set the persistent cache name that is used to construct the cache filenames. Persistent cache files are disabled by default.

zeo-blob-storage

Set the directory to be used to store blobs for a ZEO client. Defaults to using the same value as blob-storage.

zeo-shared-blob-dir

Boolean that should be 'on' if the blob dir is being shared by the ZEO server and client. Defaults to 'on'. Must be set to 'off' if this directory is not shared.

Example usage

Here's a minimal buildout that adds an extra filestorage:

[buildout]
extends = base.cfg
parts =
    filestorage
    instance

[instance]
recipe = plone.recipe.zope2instance
user = me

[filestorage]
recipe = collective.recipe.filestorage
parts =
    mystorage

By default, the location of the new filestorage will be: var/filestorage/mystorage/mystorage.fs

See above for options to override the defaults.

A setting can be modified just for one particular filestorage, by creating a new part with the filestorage_ prefix, like so:

[filestorage]
recipe = collective.recipe.filestorage
parts =
    myfirststorage
    mysecondstorage

[filestorage_mysecondstorage]
zodb-cache-size = 1000

By default, the recipe adds the extra filestorages to each plone.recipe.zope2instance part in the buildout, but you can tell it to only add it to certain parts:

[buildout]
extends = base.cfg
parts =
    filestorage
    instance1
    instance2

[instance1]
recipe = plone.recipe.zope2instance

[instance2]
recipe = plone.recipe.zope2instance

[filestorage]
recipe = collective.recipe.filestorage
zopes = instance1
parts =
    my-fs

Here is a minimal buildout including a ZEO server and two ZODB clients:

[buildout]
extends = base.cfg
parts =
    filestorage
    zeoserver
    primary
    secondary

[zeoserver]
recipe = plone.recipe.zope2zeoserver

[primary]
recipe = plone.recipe.zope2instance
zeo-client = on

[secondary]
recipe = plone.recipe.zope2instance
zeo-client = on

[filestorage]
recipe = collective.recipe.filestorage
parts =
    my-fs

As above, we can override a number of the default parameters:

[buildout]
extends = base.cfg
parts =
    filestorage
    zeoserver
    primary
    secondary

[zeoserver]
recipe = plone.recipe.zope2zeoserver

[primary]
recipe = plone.recipe.zope2instance
zeo-client = on

[secondary]
recipe = plone.recipe.zope2instance
zeo-client = on

[filestorage]
recipe = collective.recipe.filestorage
location = var/filestorage/%(fs_part_name)s/Data.fs
blob-storage = var/blobstorage-%(fs_part_name)s
zodb-cache-size = 1000
zodb-name = %(fs_part_name)s_db
zodb-mountpoint = /%(fs_part_name)s_mountpoint
zeo-address = 8101
zeo-client-cache-size = 50MB
zeo-storage = %(fs_part_name)s_storage
zeo-client-name = %(fs_part_name)s_zeostorage_name
parts =
    my-fs

By default, the recipe adds the extra filestorages to the first plone.recipe.zope2zeoserver part in the buildout, and will throw an error if there is more than one part using this recipe. However, you can override this behavior by specifying a particular ZEO part. In this case, the filestorages will only be added to the Zopes using that ZEO, by default:

[buildout]
extends = base.cfg
parts =
    filestorage
    zeoserver1
    zeoserver2
    primary
    secondary
    other-zope

[zeoserver1]
recipe = plone.recipe.zope2zeoserver
zeo-address = 8100

[zeoserver2]
recipe = plone.recipe.zope2zeoserver
zeo-address = 8101

[primary]
recipe = plone.recipe.zope2instance
zeo-client = 1
zeo-address = 8101

[secondary]
recipe = plone.recipe.zope2instance
zeo-client = 1
zeo-address = 8101

[other-zope]
recipe = plone.recipe.zope2instance
zeo-client = 1
zeo-address = 8100

[filestorage]
recipe = collective.recipe.filestorage
zeo = zeoserver2
parts =
    my-fs

Backup integration

Here's a buildout that illustrates backup integration:

[buildout]
extends = base.cfg
parts =
    filestorage
    instance
    backup

[instance]
recipe = plone.recipe.zope2instance
user = me:pass

[backup]
recipe = collective.recipe.backup>=2.7

[filestorage]
recipe = collective.recipe.filestorage
parts =
    foo
    bar
backup = backup

Running the tests

The github checkout of collective.recipe.filestorage includes a buildout which installs a script for running the tests. For this to work, you need to have the test dependencies installed:

python bootstrap.py
bin/buildout
bin/test

Alternatively, you can change the final step to ./bin/python setup.py test which will retrieve the test dependencies and run the tests.

Known issue: The tests run buildout in a separate process, so it's currently impossible to put a pdb breakpoint in the recipe and debug during the test. If you need to do this, set up another buildout which uses collective.recipe.filestorage as a development egg.

Reporting bugs or asking questions

Use the github tracker: https://github.com/collective/collective.recipe.filestorage/issues

Some old bugs are at Launchpad: https://bugs.launchpad.net/collective.buildout/