Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexes and metadata cleanup from setup_catalog #1486

Merged
merged 101 commits into from
Dec 7, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
97524e7
Cleanup getClientUID fields (added IClientBindable and ClientBindable)
xispa Nov 22, 2019
a33ee84
Cleanup getClientUID search criteria for Add form
xispa Nov 22, 2019
49a07b6
No fixme anymore
xispa Nov 22, 2019
7fe9b8e
Sanitize AnalysisRequest content type regarding Client fields
xispa Nov 22, 2019
44532fc
BatchReferenceWidgetVocabulary -> ClientBindableReferenceWidgetVocabu…
xispa Nov 22, 2019
c798735
Upgrade step for client fields reindexing
xispa Nov 22, 2019
2631317
Remove stale js that applies field filters in Sample view
xispa Nov 22, 2019
77ec02a
ajaxGetClientInfo is no longer needed
xispa Nov 22, 2019
b390684
Remove unnecessary stuff (client.ajax.ReferenceWidgetVocabulary)
xispa Nov 22, 2019
17d5a60
Make ARReport ClientBindable
xispa Nov 22, 2019
ae47c20
Omit search filter for non-bindable portal_types
xispa Nov 22, 2019
779254b
Sample edit form - some selection widgets empty (CHANGES)
xispa Nov 22, 2019
28674ee
Typo
xispa Nov 22, 2019
4283628
Fix test
xispa Nov 22, 2019
f78ac4c
Removal of unnecessary functions (provided by ClientBindable)
xispa Nov 22, 2019
1f8369d
Cleanup of code that is no longer necessary
xispa Nov 22, 2019
0472ab0
ClientBindable -> ClientAwareMixin
xispa Nov 25, 2019
71e4039
Better way of returning a Client from acquisition chain
xispa Nov 25, 2019
a423dd4
Replace clientbindable namespace
xispa Nov 25, 2019
a03c694
Use six for isinstance
xispa Nov 25, 2019
29f5d9d
Merge branch 'master' into i1476
xispa Nov 25, 2019
dbd7eda
Filter Templates field when Sample Type is selected in Sample Add form
xispa Nov 26, 2019
48ee2bb
Merge branch 'master' of github.com:senaite/senaite.core into i968
xispa Nov 27, 2019
ba94730
getSampleTypeUID index refactoring
xispa Nov 27, 2019
64b2486
Remove SampleTypeVocabulary (no longer used)
xispa Nov 28, 2019
fa962b7
Index: getSampleTypeUID --> sampletype_uids
xispa Nov 28, 2019
c0838ab
Added code header
xispa Nov 28, 2019
244dc31
Added some comments
xispa Nov 28, 2019
49cbaa5
security.public and titles separated with ", "
xispa Nov 28, 2019
0006e31
Organize imports
xispa Nov 28, 2019
bdfe66a
Add getSampleTypesTitle in ISampleTypeAwareMixin interface
xispa Nov 28, 2019
da45ff6
Remove stupid conditional
xispa Nov 28, 2019
9f006ae
Merge branch 'master' into i968
ramonski Nov 28, 2019
c317314
Redux getAccredited index and metadata
xispa Nov 28, 2019
6ac0f94
Remove getAnalyst index from setup_catalog
xispa Nov 28, 2019
05f5a15
getAvailableMethodUIDs -> method_available_uids
xispa Nov 28, 2019
721e690
Misspelling
xispa Nov 28, 2019
570b6b7
Remove getBlank index and metadata from setup_catalog
xispa Nov 28, 2019
8a84b3b
Remove index and metadata: getDuplicateVariation
xispa Nov 28, 2019
7cf7e7c
Remove getFormula index and metadata from setup_catalog
xispa Nov 28, 2019
3df7ca4
Removal of getDocumentID index and metadata from setup_catalog
xispa Nov 28, 2019
9bd34dc
Remove getInstrumentLocationName index and metadata from setup_catalog
xispa Nov 28, 2019
b7c0966
Cleanup getInstrumentTitle from setup_catalog
xispa Nov 28, 2019
4e7c0d2
Cleanup getPrice index and metadata from setup_catalog
xispa Nov 28, 2019
14b91c6
getTotalPrice --> price_total
xispa Nov 28, 2019
05c9795
getVolume --> volume
xispa Nov 28, 2019
d8ce88e
Merge branch 'master' of github.com:senaite/senaite.core into setup_i…
xispa Nov 29, 2019
11fe31e
Added IHavePrice interface
xispa Nov 29, 2019
8eb6eba
Cleanup getInstrumentTypeName from setup_catalog
xispa Nov 29, 2019
065a7ae
Remove getInstrumentType index and metadata from setup_catalog
xispa Nov 29, 2019
e6bf75c
Remove getHazardous and getManagerEmail indexes+metadata
xispa Nov 29, 2019
7ab583f
Remove getManagerPhone index and metadata from setup_catalog
xispa Nov 29, 2019
98a5eaf
Remove getManagerName index and metadata from setup_catalog
xispa Nov 29, 2019
c20d385
Remove getMethodID index from setup_catalog
xispa Nov 29, 2019
b80ad34
Remove getMaxTimeAllowed index and metadata from setup_catalog
xispa Nov 29, 2019
8ba0749
Remove getModel index and metadata from setup_catalog
xispa Nov 29, 2019
3e73b04
Remove getCalculationTitle index and metadata from setup_catalog
xispa Nov 29, 2019
42f72b6
Remove getCalculationUID index and metadata from setup_catalog
xispa Nov 29, 2019
71a81c8
getDepartmentTitle --> department_title index
xispa Nov 29, 2019
d90acf7
instrumenttype_name --> instrumenttype_title
xispa Nov 29, 2019
7ccd02d
Remove getVATAmount index and metadata from setup_catalog
xispa Nov 29, 2019
018067d
Remove getUnit index and metadata from setup_catalog
xispa Nov 29, 2019
cbfa361
getPointOfCapture --> point_of_capture
xispa Nov 29, 2019
6e79fee
Remove getSamplePointTitle index from setup_catalog
xispa Nov 29, 2019
ea90393
Remove volume index from setup_catalog
xispa Nov 29, 2019
dfd49ba
Merge branch 'master' of github.com:senaite/senaite.core into setup_i…
xispa Nov 29, 2019
701d14c
Remove getSamplePointUID index and metadata from setup_catalog
xispa Nov 29, 2019
99643e2
getSampleTypeTitle --> sampletype_title
xispa Nov 29, 2019
3fd867e
FieldIndex --> KeywordIndex
xispa Nov 29, 2019
28a6318
Indexes renaming: *_uids --> *_uid
xispa Nov 29, 2019
9fead8e
Remove index: getCategoryTitle
xispa Nov 29, 2019
69b0ccf
Code revisit and refactoring
xispa Nov 29, 2019
8c2dbbb
Remove getFullname index and metadata from setup_catalog
xispa Dec 2, 2019
a6a8f91
Remove cancellation_sate index/metadata from setup_catalog
xispa Dec 2, 2019
ad56a83
Remove getName index and metadata from setup_catalog
xispa Dec 2, 2019
df06c80
Use ReferenceWidget for Manufacturer and Location in INstrument
xispa Dec 2, 2019
6b87861
Remove getServiceUIDs index from setup_catalog
xispa Dec 2, 2019
ad6586e
Remove getServiceUID index and metadata from setup_catalog
xispa Dec 2, 2019
f954c14
Forgot to remove getServiceUID from setuphandlers
xispa Dec 2, 2019
31d4f89
Added generic listing_searchable_text for setup_catalog
xispa Dec 2, 2019
da2a64a
Comments cleanup (creation of indexers/README.md)
xispa Dec 2, 2019
bac11cb
Missed to remove getSamplePointTitle from setup_handlers
xispa Dec 2, 2019
a219af6
Remove boldies
xispa Dec 2, 2019
717a910
Fix problem with title index for Organisation-like objects
xispa Dec 2, 2019
ffe2deb
Keep labcontact in portal_catalog
xispa Dec 2, 2019
5ae70b2
Fix Client's Title() not safe_unicode
xispa Dec 2, 2019
39b9616
Remove unnecessary code from Laboratory content type
xispa Dec 2, 2019
f506b79
Remove getAnalysisCategories vocabulary from Client
xispa Dec 2, 2019
d5dba61
Fix DataLayer tests
xispa Dec 2, 2019
d92128c
Changelog
xispa Dec 2, 2019
4aada68
Added getName signature for IOrganisation
xispa Dec 2, 2019
bc81e8d
LabContacts view sorted by sortable_title
xispa Dec 3, 2019
5ce7cf2
Better searches for items from setup_catalog
xispa Dec 3, 2019
f1280ae
Remove unused Schema fields from Instrument
xispa Dec 3, 2019
c8c218b
Additional fields for listing_searchable_text
xispa Dec 3, 2019
76da915
Make WorksheetTemplate and Method to implement IHaveInstrument
xispa Dec 3, 2019
b7b7e50
Better handling of lists in listing_searchable_text
xispa Dec 3, 2019
7e18767
Remove ajaxGetClients
xispa Dec 3, 2019
520e020
Better comment
xispa Dec 3, 2019
eca310b
getCategoryUID index --> category_uid
xispa Dec 3, 2019
481de70
Simplify to_searchable_metadata function
xispa Dec 3, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Changelog

**Changed**

- #1486 Clean-up of indexes and metadata from `setup_catalog`

**Removed**

Expand Down
35 changes: 32 additions & 3 deletions CODE_CONVENTIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,33 @@ any add-ons for `senaite` too.
Names for catalog indexes don't follow the `CamelCase` naming convention, rather
all in lowercase and words separated by `_`:

- Wrong: `getSampleTypeUID`
- Good: `sampletype_uids`
- Bad: `getSampleTypeUID`
- Good: `sampletype_uid`

#### Plural and singular forms

Indexes will be written in singular form. There are a few exceptions, mostly
those that come from `Plone` or `Zope` (e.g. `allowedRolesAndUsers`). Quite
frequently, the plural form is used wrongly because the name of the meta-type
index to use leads us to think about the plural form: e.g. `KeywordIndex`, that
indexes a sequence of keywords. Is better to think about how the searches
against the index work.

For instance, for a given object, a `KeywordIndex` will store a sequence of
keywords, but if we do a search for single or multiple keywords against this
index, the search will only return those items that have at least one of the
keywords. And if there are multiple keyword matches for same item, the system
will only return the item once. Since we can query for any index (`FieldIndex`,
`KeywordIndex`, etc.) using a list, it does not make sense to use the plural
form. In fact, if you are forced to add an index in plural form because a given
index with same name, but in singular already exists, probably the index in
singular is a `FieldIndex`, that don't allow you to store multiple values. In
such case, the best approach is to change the meta-type of the existing index
from `FieldIndex` to `KeywordIndex`.

- Bad: `sampletype_titles`
- Good: `sampletype_title`


### Metadata fields naming

Expand All @@ -22,6 +47,10 @@ convention. Since one would expect the name of the metadata field to match
with the name of the function from the object, we keep same convention. This
might change in future when we move to `Dexterity`.

- Wrong: `SampleTypeUID`
- Bad: `SampleTypeUID`
- Good: `getSampleTypeUID`

#### Plural and singular forms

For metadata fields, use plural forms when the field returns a list and use
singular when the field returns a single value.
12 changes: 7 additions & 5 deletions bika/lims/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1320,13 +1320,15 @@ def to_searchable_text_metadata(value):
if isinstance(value, (bool)):
return u""
if isinstance(value, (list, tuple)):
for v in value:
return to_searchable_text_metadata(v)
if isinstance(value, (dict)):
for k, v in value.items():
return to_searchable_text_metadata(v)
values = map(to_searchable_text_metadata, value)
values = filter(None, values)
return " ".join(values)
if isinstance(value, dict):
return to_searchable_text_metadata(value.values())
if is_date(value):
return value.strftime("%Y-%m-%d")
if is_at_content(value):
return to_searchable_text_metadata(get_title(value))
if not isinstance(value, basestring):
value = str(value)
return safe_unicode(value)
Expand Down
21 changes: 18 additions & 3 deletions bika/lims/browser/accreditation.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,13 @@
# Some rights reserved, see README and LICENSE.

from Products.CMFPlone.utils import safe_unicode
from plone.app.content.browser.interfaces import IFolderContentsView
from zope.interface import implements

from bika.lims import api
from bika.lims import bikaMessageFactory as _
from bika.lims.controlpanel.bika_analysisservices import AnalysisServicesView
from bika.lims.utils import t
from plone.app.content.browser.interfaces import IFolderContentsView
from zope.interface import implements


class AccreditationView(AnalysisServicesView):
Expand All @@ -34,7 +36,6 @@ def __init__(self, context, request):
self.contentFilter = {
'portal_type': 'AnalysisService',
'sort_on': 'sortable_title',
'getAccredited': True,
'is_active': True}
self.context_actions = {}
self.title = self.context.translate(_("Accreditation"))
Expand Down Expand Up @@ -102,3 +103,17 @@ def selected_cats(self, items):
if 'category' in item and item['category'] not in cats:
cats.append(item['category'])
return cats

def search(self, searchterm="", ignorecase=True):
# Boil out those that are not accredited
# There is no need to keep a `getAccredited` index in the catalog only
# for this view. Since we don't expect a lot of items from this content
# type (AnalysisService), is fine to wake-up them
brains = super(AccreditationView, self).search(searchterm, ignorecase)

def is_accredited(brain):
obj = api.get_object(brain)
return obj.getAccredited()

return filter(is_accredited, brains)

8 changes: 4 additions & 4 deletions bika/lims/browser/analysisrequest/add2.py
Original file line number Diff line number Diff line change
Expand Up @@ -538,7 +538,7 @@ def get_services(self, poc="lab"):
bsc = api.get_tool("bika_setup_catalog")
query = {
"portal_type": "AnalysisService",
"getPointOfCapture": poc,
"point_of_capture": poc,
"is_active": True,
"sort_on": "sortable_title",
}
Expand Down Expand Up @@ -1018,18 +1018,18 @@ def get_sampletype_info(self, obj):
# Display Sample Points that have this sample type assigned plus
# those that do not have a sample type assigned
"SamplePoint": {
"sampletype_uids": [sample_type_uid, None],
"sampletype_uid": [sample_type_uid, None],
"getClientUID": [client_uid, ""],
},
# Display Specifications that have this sample type assigned only
"Specification": {
"sampletype_uids": sample_type_uid,
"sampletype_uid": sample_type_uid,
"getClientUID": [client_uid, ""],
},
# Display AR Templates that have this sample type assigned plus
# those that do not have a sample type assigned
"Template": {
"sampletype_uids": [sample_type_uid, None],
"sampletype_uid": [sample_type_uid, None],
"getClientUID": [client_uid, ""],
}
}
Expand Down
1 change: 0 additions & 1 deletion bika/lims/browser/analysisrequest/templates/ar_add2.pt
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,6 @@
<tal:services tal:repeat="service python:services[category_title];">
<tr tal:define="service_uid service/UID;
service_id service/getId;
service_acc service/getAccredited;
service_keyword service/getKeyword;
service_title service/Title;"
tal:attributes="id string:${service_id};
Expand Down
73 changes: 7 additions & 66 deletions bika/lims/browser/clientfolder.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,19 @@
# Some rights reserved, see README and LICENSE.

import collections
import json

from Products.CMFCore.permissions import ModifyPortalContent
from bika.lims import api, logger
from plone.app.content.browser.interfaces import IFolderContentsView
from zope.interface import implements

from bika.lims import bikaMessageFactory as _
from bika.lims.browser import BrowserView
from bika.lims.browser.bika_listing import BikaListingView
from bika.lims.permissions import AddClient
from bika.lims.permissions import ManageAnalysisRequests
from bika.lims.utils import (check_permission, get_email_link, get_link,
get_registry_value)
from plone import protect
from plone.app.content.browser.interfaces import IFolderContentsView
from zope.interface import implements
from bika.lims.utils import check_permission
from bika.lims.utils import get_email_link
from bika.lims.utils import get_link
from bika.lims.utils import get_registry_value


class ClientFolderContentsView(BikaListingView):
Expand Down Expand Up @@ -199,61 +198,3 @@ def client_match(client, search_term):
if search_term in client.Description().lower():
return True
return False


class ajaxGetClients(BrowserView):
"""Vocabulary source for jquery combo dropdown box
"""

def __call__(self):
protect.CheckAuthenticator(self.request)
searchTerm = self.request.get("searchTerm", "").lower()
page = self.request.get("page", 1)
nr_rows = self.request.get("rows", 20)
sort_order = self.request.get("sord") or "ascending"
sort_index = self.request.get("sidx") or "sortable_title"

if sort_order == "desc":
sort_order = "descending"

# Use the catalog to speed things up and also limit the results
catalog = api.get_tool("portal_catalog")
catalog_query = {
"portal_type": "Client",
"review_state": "active",
"sort_on": sort_index,
"sort_order": sort_order,
"sort_limit": 500
}
# Inject the searchTerm to narrow the results further
if searchTerm:
catalog_query["SearchableText"] = searchTerm
logger.debug("ajaxGetClients::catalog_query=%s" % catalog_query)
brains = catalog(catalog_query)
rows = []

for brain in brains:
client = brain.getObject()
# skip clients where the search term does not match
if searchTerm and not client_match(client, searchTerm):
continue
rows.append(
{
"ClientID": client.getClientID(),
"Title": client.Title(),
"ClientUID": client.UID(),
}
)

pages = len(rows) / int(nr_rows)
pages += divmod(len(rows), int(nr_rows))[1] and 1 or 0
ret = {
"page": page,
"total": pages,
"records": len(rows),
"rows": rows[
(int(page) - 1) * int(nr_rows): int(page) * int(nr_rows)
]
}

return json.dumps(ret)
8 changes: 0 additions & 8 deletions bika/lims/browser/clientfolder.zcml
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,4 @@
layer="bika.lims.interfaces.IBikaLIMS"
/>

<browser:page
for="*"
name="getClients"
class="bika.lims.browser.clientfolder.ajaxGetClients"
permission="zope.Public"
layer="bika.lims.interfaces.IBikaLIMS"
/>

</configure>
4 changes: 2 additions & 2 deletions bika/lims/browser/department/labcontacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,12 @@ def __init__(self, context, request):
self.context_actions = {}
self.contentFilter = {
'portal_type': 'LabContact',
'sort_on': 'getFirstname',
'sort_on': 'sortable_title',
'sort_order': 'ascending'
}
self.columns = {
'Fullname': {'title': _('Name'),
'index': 'getFullname'},
'index': 'sortable_title'},
'Department': {'title': _('Department')},
'BusinessPhone': {'title': _('Phone')},
'MobilePhone': {'title': _('Mobile Phone')},
Expand Down
3 changes: 0 additions & 3 deletions bika/lims/browser/js/bika.lims.loader.js
Original file line number Diff line number Diff line change
Expand Up @@ -131,9 +131,6 @@ window.bika.lims.controllers = {
".portaltype-worksheetfolder":
['WorksheetFolderView'],

".portaltype-worksheet.template-add_analyses":
['WorksheetAddAnalysesView'],

".portaltype-worksheet.template-add_blank":
['WorksheetAddQCAnalysesView'],

Expand Down
Loading