Support Multiple Catalogs for Dexterity Contents

[ramonski]

Description of the issue/feature this PR addresses

This PR implements an IIndexQueueProcessor to multiplex Dexterity indexing to multiple catalogs

Usage

Add the bika.lims.interfaces.IMultiCatalogBehavior to the Dexterity type config:

  <!-- Dexterity behaviours for this type -->
  <property name="behaviors">
    <element value="bika.lims.interfaces.IAutoGenerateID"/>
    <element value="bika.lims.interfaces.IMultiCatalogBehavior"/>
    <element value="plone.app.dexterity.behaviors.metadata.IBasic"/>
    <element value="plone.app.referenceablebehavior.referenceable.IReferenceable" />
  </property>

Add the supported catalogs into an _catalogs attribute of your Dexterity content:

from bika.lims.catalog import SETUP_CATALOG
from plone.dexterity.content import Item
from plone.supermodel import model
from zope.interface import implementer


class IDynamicAnalysisSpec(model.Schema):
    """Dynamic Analysis Specification
    """

    specs_file = namedfile.NamedBlobFile(
        title=_(u"Specification File"),
        description=_(u"Only Excel files supported"),
        required=True)


@implementer(IDynamicAnalysisSpec)
class DynamicAnalysisSpec(Item):
    """Dynamic Analysis Specification
    """
    _catalogs = [SETUP_CATALOG]

The catalog multiplexer will automatically index the content in the requested catalogs.

Current behavior before PR

Dexterity contents are only indexed in portal_catalog

Desired behavior after PR is merged

Dexterity contents are indexed in portal_catalog and additional catalogs

--
I confirm I have tested this PR thoroughly and coded it according to PEP8
and Plone's Python styleguide standards.

[xispa]

Small performance bit

[xispa]

Wouldn't the following be easier?:

catalogs = getattr(obj, "_catalogs", []) + REQUIRED_CATALOGS
catalogs = list(set(catalogs))
return map(api.get_tool, catalogs)

[ramonski]

I think the current approach is more readable and performant, because in your approach the list hast to be iterated multiple times to generate first a set and convert it then to a list.
But indeed, it would be interesting what approach is faster...

... 5 minutes later ...

Here comes the proof:

REQUIRED_CATALOGS = [
    "auditlog_catalog",
]

def v1(obj):
    catalogs = getattr(obj, "_catalogs", [])
    for rc in REQUIRED_CATALOGS:
        if rc in catalogs:
            continue
        catalogs.append(rc)
    return catalogs

def v2(obj):
     catalogs = getattr(obj, "_catalogs", []) + REQUIRED_CATALOGS
     catalogs = list(set(catalogs))
     return catalogs

class Content(object):
     _catalogs = ["setup_catalog"]

obj = Content()

if __name__ == '__main__':
    from timeit import timeit
    print timeit("v1(obj)", setup="from __main__ import obj, v1")
    print timeit("v2(obj)", setup="from __main__ import obj, v2")

Output:

0.572149038315
0.9786028862

[xispa]

pheew!

[xispa]

If attributes have been manually specified, but no matches with current catalog are found, the object will be reindexed for all indexes. I would rather skip this case for better performance:

indexes = set(catalog.indexes()).intersection(attributes)
if attributes and not indexes:
    continue

logger.info(
    "CatalogMultiplexProcessor::indexObject:catalog={} url={}"
    .format(catalog.id, url))
catalog.catalog_object(obj, url, idxs=list(indexes))

[ramonski]

agree, will change that

[xispa]

Awesome!