State of Reference Management (done)

This feature will be part of Silva 2.3.

Using references in default Silva content

This new references system will be used in Silva in order to express references between:

  • A Silva [relative] Link and its target,
  • A Silva Ghost Folder and its target,
  • A Silva Ghost and its target,
  • A Silva Indexer and its indexed documents (already partially done),
  • A Silva Document and its included Silva Image,
  • A Silva Document and its included Silva Link (does this is possible ?),
  • A Silva Permanent Redirect Link and its target,
  • Complete me ?

Storing references

References to content will be stored as an object. This object will support the operations set, get, query and url to set and retrieve the target of the reference.

This object will maintain a working reference to the target content even if it is moving around. As an intern implementation, they will use intids to identify targets, so this won't be an issue.

Formulator, formlib and z3c.form fields will be available in order to edit those references objects:

  • You will be able to browse Silva content in order in a popup window in order to select a target. An upper option might let you search a content using a query to the catalog.
  • You will be able to type a relative path in a text field in order to select a target.

All Silva code will use those widgets to manage reference if possible (and not rebuild its own custom version each time).

When rendered to the user, a URL to the target of the reference should always be used (and never an internal ID).

Managing references integrity

Intids doesn't trigger any special actions when a object is removed. In order to ensure an integrity of the current references, a new service will be added: service_integrity. It will index, with the help of an IOBTree which content is referred by which other content.

When an object is going to be deleted, this service will be checked in order to verify if any existing reference will be broken by this action.

If a reference might be broken, an error will be raised, and intercepted by the interface: it will display a warning message reporting to the user that the following content he is trying to delete is referred in other ones. Depending of the target (policy setting) the user will have possibility to:

  • Not be able to do anything,
  • To be able to delete the current content anyway,
  • To be able to delete the current content and references target at the same time.

This reference checking might be implemented in a clever to support recursive deletion: if a reference exist to an another content which is scheduled to be deleted as well, no warning should be issued.

This service will provides a human interfaces:

  • To assess contents containing broken references.

This service will provides an API for Silva developers to query the existing relations between contents:

  • Providing a path and an content, a method will check that all references used by this content are located inside the given path. This will help exporting Silva content to Silva XML,
  • Providing a content, a method will check if the content is linked to any other content. This will make possible in the interface to report asset not referred by any other content.

References and Versioned Content

In order to be able to delete unused assets, by default, references to versioned content will index in service_integrity references to the currently published version and the current editable version. Of course, broken references will appear if you republish an previously closed version of a content, which refer to deleted content.

Exporting Silva Content in Silva XML

When exporting Silva content into Silva XML (a zip that you can import later on), references will be checked for any references located outside of the exported content tree. If any references are found, an error will be reported including the list of those references.

Inside of the Silva XML, all references will be converted to global paths where the root (/) is the exported Silva content. When importing new Silva XML, those paths will be converted back to  references objects.

In order to be able to import archives from previous Silva version, a parameter origin will be added to the importing API in order to specify which was the original path of the content in the original site where it have be exported. This will make possible to recompute references correctly.

Encoding Links in Silva Documents


The direction SRM will take is to use this intid service to translate resolvable zope paths within the same vhost to intids, and store these in the silvaxml.  When the link is translated for output, either for public viewing or for editing in kupu for the forms editor, the intid will be translated to a path relative to the vhost root.

There are other types of urls which cannot resolve to intids:

  1. relative paths which cannot resolve to zope objects
  2. absolute urls
  3. anchors/bookmarks, etc.

Because of this, it will be necessary to distinguish between intids and everything else.  When paths are encoded for storage in silvaxml, a prefix will be added to them, either intid: or path:.  This will allow the rendering machinery to resolve the intids to objects, or act on paths.  It is assumed that links which do not have a prefix are paths.

Intid's will be stored in the silvaxml along with the path to the object, in a form like this: "intid:<intid>:<path> .  This is to cover the case of references which have been removed.  If the silvaxml has just an intid, it will be impossible to learn where the intid used to go.  If the path is also stored in the silvaxml, at least that will give editors an idea of where the link went to.

Also note that incoming absolute urls for storage in silvaxml that have the same host as the vhost (e.g. copy/paste the location) should be converted to paths.  But, how can you take into account http vs https?


The change to link encoding will require an expensive content conversion that will be run as part of an upgrade. Missing target will be reported in the logs, and viewable through the service_integrity.