Search filter component extension

Add an extension to your component manifest as follows:

  <extension type="plugin">
   
        <filename>extension_script.py</filename>
        <provides>SEARCH_FILTER_PLUGIN</provides>
        <provides_for>
            <searchtype>SEARCHTYPE</searchtype>
        </provides_for>
    </extension>

The idmlib.plugins.searchfilter.criteria component contains classes pre-configured with the available search criteria for many common search engines. Look for the class with a _search_type matching the engine you defined in the manifest.

List criteria are a more limited interface that allows you to explicitly show (include) or hide (exclude) uniquely identified objects from a particular search. Although more limited, this method is much more performance efficient if the logic required to filter the objects using standard search criteria would be very complex.

Only certain search types support list criteria (generally only those where it is possible to uniquely identify every available object the search applies to), a given search criteria class supports list criteria if it has _supports_lists as True. When passing criteria to list criteria methods, you must provide a value for all the keys in _key_fields; values for any other fields will be ignored.

To get an object you can pass to the list criteria methods you should use the instance method of your criteria class. You can pass the values for the required fields either as keyword arguments to instance, or by setting them as attributes afterwards.

The two most important list criteria methods are include and exclude. Objects passed to include will be shown, and objects passed to exclude will be hidden. Here’s a short example:

from idmlib.components.extension import ExtPlugin
from idmlib.plugins.searchfilter import SearchFilter
from idmlib.plugins.searchfilter.criteria import CriteriaRole
class ListSearchFilter(ExtPlugin):
    def __init__(self, plugin: SearchFilter, state: dict):
        super().__init__()
        self.plugin = plugin
        self.state = state
    def process(self):
        for role in {"IT", "ADMIN", "MAINTENANCE"}:
   
            self.plugin.include(CriteriaRole.instance(roleid=role))
        for role in {"HR", "CUSTOMER\_SERVICE"}:
            self.plugin.exclude(CriteriaRole.instance(roleid=role)) 

Search criteria allow you to filter what is visible to the user using search queries. Individual criteria are simple: A field to compare, a comparison to use, and a value to compare against. These individual criteria can be combined to build more complex queries, but the combination is limited, groups of criteria will combine with AND, and different groups will then be combined with OR. This limitation requires that you break down complex logic and think carefully about the result you want the sum of your criteria to have. Additionally, while powerful, search queries are not incredibly performant, if you find yourself emitting several hundred criteria for a particular search engine, you may want to look at refactoring to use list criteria (if supported) as filtering on that many criteria will start to slow down the UI, and hurt the experience of the end user.

Search criteria are created by operating on the provided criteria class, or by calling methods named after the particular comparison you want to use; for example: equals/==, less_than/before/<, is_/>> and so on.

from idmlib.components.extension import ExtPlugin
from idmlib.plugins.searchfilter import SearchFilter
from idmlib.plugins.searchfilter.criteria import CriteriaUser
class MySearchPlugin(ExtPlugin):
    def __init__(self, plugin: SearchFilter, state: dict):
        super().__init__()
        self.plugin = plugin
        self.state = state
    def process(self):
        for target in {"AD", "AD_TWO", "EXTERNAL"}:
            for group in {"GROUP_ONE", "GROUP_TWO"}:
   
                self.plugin.builder.get(
                    f"{target}-{group}-filter"
                ).add(
                    CriteriaUser.hostid == target,
                    CriteriaUser.groupid == group,
                    CriteriaUser.managerid == "ImportantManager",
                ) 

The resulting filter will display only Users where:

Or 6 filters (AD-GROUP_ONE-filter, AD-GROUP_TWO-filter, AD_TWO-GROUP_ONE-filter, ...) each containing 3 criteria, for a total of 18 criteria.

The FilterBuilder is the core of how to build search filters. It provides a single, restrictive, validated method for constructing search filters.

A FilterBuilder with the correct search criteria validation information is created for you at plugin.builder during the creation of the SearchFilter plugin object.

The main logic of the FilterBuilder works as follows:

FilterBuilder has the following public methods:

A FilterEditor is a collection of SearchExpressions, with public methods for adding/removing/replacing individual expressions, and validation logic to ensure an expression is valid for a given search. A SearchExpression is an individual criteria comparison, the result of using an operator on a SearchKey , more details later, but the below examples should be good enough to get an idea of how to work with them.

Any public method on a FilterEditor returns the same editor, so you can ’chain’ operations together or store the editor for later use, add and remove also support any number of arguments, allowing you to pass multiple SearchExpressions to them.

It has the following public methods:

List criteria passed to these methods are grouped with a ’tag’ (very similar to the builder’s filter groups). Criteria in different tags is unioned together during plugin close. The methods can accept multiple list criteria at once, making it easy to add a large number of criteria at one time.

Two new methods were added, one for inclusion, and another for exclusion, but both function the same. These methods are designed for the use case where one component wants to reduce/further restrict the list criteria added by another, but does not want to have to re-do the same calculations as the other component in order to do so.

The _and methods are basically a set intersection (the criteria in the tag is reduced to the criteria in the input and in the tag).

Example:

If add_if_empty is True, and you call an _and method on an empty tag, then instead of resulting in the tag remaining empty (since it’s a set intersection), the criteria is instead added to the tag as though you had just called include/exclude themselves. In the above example, provided we always knew that both involved components would always add some number of list criteria, both could instead call include_and with add_if_empty set to True, allowing the component priorities to be re-ordered without any logic needing to be changed.

Example: (equivalent to previous example)

This is why tagging is useful, as without it it would quickly be very difficult for multiple components (operating on the same search) to take advantage of this feature if they could not adequately separate what criteria they cared about.

A SearchExpression is a Python dataclass representation of a single filter group in the KVG, minus the position and operator. It is read only, and cannot be altered once created. It has a single public method: negate() which returns a new expression with the negated value flipped, it is equivalent to doing ~SearchExpression, since it also implements the invert operator. The value of SearchExpression().comparator must be a value from the SearchExpression.Comparator enum, however you should never create expressions directly outside of testing.

A SearchKey can be thought of as a generator class for SearchExpressions. It will generate the expected SearchExpression when used with a comparison operator and a value, the SearchKey should always be the left side of the comparison. SearchKeys can be initialized a few ways:

To access nested attributes of complex SearchKeys that contain @ or - use _AT_ and _DASH_ instead. Example: S earchKey(’accountattr’, complex=True)._AT_accountEnabled

SearchKeys support the following operators/methods:

SearchCriteria is the base class for capturing the available SearchKeys for a given engine. Commonly used Criteria classes can be found in idmlib.plugins.searchfilter.criteria. A SearchCriteria class is a collection of SearchKeys, representing a single search type. However, it has a few interesting properties in order to support the use of list criteria:

Additionally, the following property helps in cases where some keys may be defined in the plugin input, but the class has not yet been updated with the new keys: