Botorch with cardinality constraint via sampling (#301)

(edited by @AdrianSosic)

This PR adds support for cardinality constraints to
`BotorchRecommender`. The core idea is to tackle the problem in an
exhaustive search like manner, i.e. by
* enumerating the possible combinations of in-/active parameters
dictated by the cardinality constraints
* optimizing the corresponding restricted subspaces, where the
cardinality constraint can then be simply removed since the in-/active
sets are fixed within these subspaces
* aggregating the optimization results of the individual subspaces into
a single recommendation batch.

The PR implements two mechanisms for determining the configuration of
inactive parameters:
- When the combinatorial list of possible inactive parameter
configurations is not too large, we iterate the full list
- otherwise, a fixed amount of inactive parameter configurations is
randomly selected

The current aggregation step is to simply optimize all subspaces
independently of each other and then return the batch from the subspace
where the highest acquisition value is achieved. This has the
side-effect that the set of inactive parameters is the same across the
entire recommendation batch. This can be a desirable property in many
use cases but potentially higher acquisition values can be obtained by
altering the in-/activity sets across the batch. A simple way to achieve
this (though out of scope for this PR) is by generalizing the sequential
greedy principle to multiple subspaces.

### Out of scope
* Fulfilling cardinality constraints by passing them in suitable form as
nonlinear constraints to the optimizer
* Sequential greedy optimization to achieve varying in-/activity sets
(see explanation above)

Author: AdrianSosic

CHANGELOG.md

@@ -19,6 +19,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `is_multi_output` attribute to `Objective`
 - `supports_multi_output` attribute/property to `Surrogate`/`AcquisitionFunction`
 - `n_outputs` property to `Objective`
+- `ContinuousCardinalityConstraint` is now compatible with `BotorchRecommender`
+- A `MinimumCardinalityViolatedWarning` is triggered when minimum cardinality
+  constraints are violated
+- Attribute `max_n_subspaces` to `BotorchRecommender`, allowing to control
+  optimization behavior in the presence of cardinality constraints
+- Utilities `inactive_parameter_combinations` and`n_inactive_parameter_combinations` 
+  to both `ContinuousCardinalityConstraint`and `SubspaceContinuous` for iterating
+  over cardinality-constrained parameter sets
+- Attribute `relative_threshold` and method `get_absolute_thresholds` to 
+  `ContinuousCardinalityConstraint` for handling inactivity ranges
+- Utilities `activate_parameter` and `is_cardinality_fulfilled` for enforcing and
+  validating cardinality constraints
+- Utility `is_inactive` for determining if parameters are inactive
 
 ### Changed
 - Acquisition function indicator `is_mc` has been removed in favor of new indicators 

README.md

@@ -43,6 +43,7 @@ The following provides a non-comprehensive overview:
 - 🎭 Hybrid (mixed continuous and discrete) spaces
 - 🚀 Transfer learning: Mix data from multiple campaigns and accelerate optimization
 - 🎰 Bandit models: Efficiently find the best among many options in noisy environments (e.g. A/B Testing)
+- 🔢 Cardinality constraints: Control the number of active factors in your design
 - 🌎 Distributed workflows: Run campaigns asynchronously with pending experiments
 - 🎓 Active learning: Perform smart data acquisition campaigns
 - ⚙️ Custom surrogate models: Enhance your predictions through mechanistic understanding

baybe/constraints/continuous.py

@@ -4,11 +4,13 @@
 
 import gc
 import math
-from collections.abc import Collection, Sequence
+from collections.abc import Collection, Iterator, Sequence
+from itertools import combinations
+from math import comb
 from typing import TYPE_CHECKING, Any
 
 import numpy as np
-from attr.validators import in_
+from attr.validators import gt, in_, lt
 from attrs import define, field
 
 from baybe.constraints.base import (
@@ -17,6 +19,7 @@
     ContinuousNonlinearConstraint,
 )
 from baybe.parameters import NumericalContinuousParameter
+from baybe.utils.interval import Interval
 from baybe.utils.numerical import DTypeFloatNumpy
 from baybe.utils.validation import finite_float
 
@@ -138,6 +141,40 @@ class ContinuousCardinalityConstraint(
 ):
     """Class for continuous cardinality constraints."""
 
+    relative_threshold: float = field(
+        default=1e-3, converter=float, validator=[gt(0.0), lt(1.0)]
+    )
+    """A relative threshold for determining if a value is considered zero.
+
+    The threshold is translated into an asymmetric open interval around zero via
+    :meth:`get_absolute_thresholds`.
+
+    **Note:** The interval induced by the threshold is considered **open** because
+    numerical routines that optimize parameter values on the complementary set (i.e. the
+    value range considered "nonzero") may push the numerical value exactly to the
+    interval boundary, which should therefore also be considered "nonzero".
+    """
+
+    @property
+    def n_inactive_parameter_combinations(self) -> int:
+        """The number of possible inactive parameter combinations."""
+        return sum(
+            comb(len(self.parameters), n_inactive_parameters)
+            for n_inactive_parameters in self._inactive_set_sizes()
+        )
+
+    def _inactive_set_sizes(self) -> range:
+        """Get all possible sizes of inactive parameter sets."""
+        return range(
+            len(self.parameters) - self.max_cardinality,
+            len(self.parameters) - self.min_cardinality + 1,
+        )
+
+    def inactive_parameter_combinations(self) -> Iterator[frozenset[str]]:
+        """Get an iterator over all possible combinations of inactive parameters."""
+        for n_inactive_parameters in self._inactive_set_sizes():
+            yield from combinations(self.parameters, n_inactive_parameters)
+
     def sample_inactive_parameters(self, batch_size: int = 1) -> list[set[str]]:
         """Sample sets of inactive parameters according to the cardinality constraints.
 
@@ -176,6 +213,34 @@ def sample_inactive_parameters(self, batch_size: int = 1) -> list[set[str]]:
 
         return inactive_params
 
+    def get_absolute_thresholds(self, bounds: Interval, /) -> Interval:
+        """Get the absolute thresholds for a given interval.
+
+        Turns the relative threshold of the constraint into absolute thresholds
+        for the considered interval. That is, for a given interval ``(a, b)`` with
+        ``a <= 0`` and ``b >= 0``, the method returns the interval ``(r*a, r*b)``,
+        where ``r`` is the relative threshold defined by the constraint.
+
+        Args:
+            bounds: The specified interval.
+
+        Returns:
+            The absolute thresholds represented as an interval.
+
+        Raises:
+            ValueError: When the specified interval does not contain zero.
+        """
+        if not bounds.contains(0.0):
+            raise ValueError(
+                f"The specified interval must contain zero. "
+                f"Given: {bounds.to_tuple()}."
+            )
+
+        return Interval(
+            lower=self.relative_threshold * bounds.lower,
+            upper=self.relative_threshold * bounds.upper,
+        )
+
 
 # Collect leftover original slotted classes processed by `attrs.define`
 gc.collect()

baybe/constraints/utils.py

@@ -0,0 +1,49 @@
+"""Constraint utilities."""
+
+import numpy as np
+import pandas as pd
+
+from baybe.parameters.utils import is_inactive
+from baybe.searchspace import SubspaceContinuous
+
+
+def is_cardinality_fulfilled(
+    df: pd.DataFrame,
+    subspace_continuous: SubspaceContinuous,
+    *,
+    check_minimum: bool = True,
+    check_maximum: bool = True,
+) -> bool:
+    """Validate cardinality constraints in a dataframe of parameter configurations.
+
+    Args:
+        df: The dataframe to be checked.
+        subspace_continuous: The subspace spanned by the considered parameters.
+        check_minimum: If ``True``, minimum cardinality constraints are validated.
+        check_maximum: If ``True``, maximum cardinality constraints are validated.
+
+    Returns:
+        ``True`` if all cardinality constraints are fulfilled, ``False`` otherwise.
+    """
+    for c in subspace_continuous.constraints_cardinality:
+        # Get the activity thresholds for all parameters
+        cols = df[c.parameters]
+        thresholds = {
+            p.name: c.get_absolute_thresholds(p.bounds)
+            for p in subspace_continuous.get_parameters_by_name(c.parameters)
+        }
+        lower_thresholds = [thresholds[p].lower for p in cols.columns]
+        upper_thresholds = [thresholds[p].upper for p in cols.columns]
+
+        # Count the number of active values per dataframe row
+        inactives = is_inactive(cols, lower_thresholds, upper_thresholds)
+        n_zeros = inactives.sum(axis=1)
+        n_active = len(c.parameters) - n_zeros
+
+        # Check if cardinality is violated
+        if check_minimum and np.any(n_active < c.min_cardinality):
+            return False
+        if check_maximum and np.any(n_active > c.max_cardinality):
+            return False
+
+    return True

baybe/constraints/validation.py

@@ -8,8 +8,14 @@
 from baybe.constraints.discrete import (
     DiscreteDependenciesConstraint,
 )
+from baybe.parameters import NumericalContinuousParameter
 from baybe.parameters.base import Parameter
 
+try:  # For python < 3.11, use the exceptiongroup backport
+    ExceptionGroup
+except NameError:
+    from exceptiongroup import ExceptionGroup
+
 
 def validate_constraints(  # noqa: DOC101, DOC103
     constraints: Collection[Constraint], parameters: Collection[Parameter]
@@ -26,6 +32,8 @@ def validate_constraints(  # noqa: DOC101, DOC103
         ValueError: If any discrete constraint includes a continuous parameter.
         ValueError: If any discrete constraint that is valid only for numerical
             discrete parameters includes non-numerical discrete parameters.
+        ValueError: If any parameter affected by a cardinality constraint does
+            not include zero.
     """
     if sum(isinstance(itm, DiscreteDependenciesConstraint) for itm in constraints) > 1:
         raise ValueError(
@@ -41,6 +49,9 @@ def validate_constraints(  # noqa: DOC101, DOC103
     param_names_discrete = [p.name for p in parameters if p.is_discrete]
     param_names_continuous = [p.name for p in parameters if p.is_continuous]
     param_names_non_numerical = [p.name for p in parameters if not p.is_numerical]
+    params_continuous: list[NumericalContinuousParameter] = [
+        p for p in parameters if isinstance(p, NumericalContinuousParameter)
+    ]
 
     for constraint in constraints:
         if not all(p in param_names_all for p in constraint.parameters):
@@ -78,6 +89,11 @@ def validate_constraints(  # noqa: DOC101, DOC103
                 f"Parameter list of the affected constraint: {constraint.parameters}."
             )
 
+        if isinstance(constraint, ContinuousCardinalityConstraint):
+            validate_cardinality_constraint_parameter_bounds(
+                constraint, params_continuous
+            )
+
 
 def validate_cardinality_constraints_are_nonoverlapping(
     constraints: Collection[ContinuousCardinalityConstraint],
@@ -98,3 +114,50 @@ def validate_cardinality_constraints_are_nonoverlapping(
                 f"cannot share the same parameters. Found the following overlapping "
                 f"parameter sets: {s1}, {s2}."
             )
+
+
+def validate_cardinality_constraint_parameter_bounds(
+    constraint: ContinuousCardinalityConstraint,
+    parameters: Collection[NumericalContinuousParameter],
+) -> None:
+    """Validate that all parameters of a continuous cardinality constraint include zero.
+
+    Args:
+        constraint: A continuous cardinality constraint.
+        parameters: A collection of parameters, including those affected by the
+            constraint.
+
+    Raises:
+        ValueError: If one of the affected parameters does not include zero.
+        ExceptionGroup: If several of the affected parameters do not include zero.
+    """
+    exceptions = []
+    for name in constraint.parameters:
+        try:
+            parameter = next(p for p in parameters if p.name == name)
+        except StopIteration as ex:
+            raise ValueError(
+                f"The parameter '{name}' referenced by the constraint is not contained "
+                f"in the given collection of parameters."
+            ) from ex
+
+        if not parameter.is_in_range(0.0):
+            exceptions.append(
+                ValueError(
+                    f"The bounds of all parameters affected by a constraint of type "
+                    f"'{ContinuousCardinalityConstraint.__name__}' must include zero, "
+                    f"but the bounds of parameter '{name}' are "
+                    f"{parameter.bounds.to_tuple()}, which may indicate unintended "
+                    f"settings in your parameter definition. "
+                    f"A parameter whose value range excludes zero trivially "
+                    f"increases the cardinality of the resulting configuration by one. "
+                    f"Therefore, if your parameter definitions are all correct, "
+                    f"consider excluding the parameter from the constraint and "
+                    f"reducing the cardinality limits by one accordingly."
+                )
+            )
+
+    if exceptions:
+        if len(exceptions) == 1:
+            raise exceptions[0]
+        raise ExceptionGroup("Invalid parameter bounds", exceptions)

baybe/exceptions.py

@@ -33,6 +33,10 @@ def __str__(self):
         return self.message
 
 
+class MinimumCardinalityViolatedWarning(UserWarning):
+    """Minimum cardinality constraints are violated."""
+
+
 ##### Exceptions #####
 
 
@@ -63,6 +67,10 @@ class IncompatibleArgumentError(IncompatibilityError):
     """An incompatible argument was passed to a callable."""
 
 
+class InfeasibilityError(Exception):
+    """An optimization problem has no feasible solution."""
+
+
 class NotEnoughPointsLeftError(Exception):
     """
     More recommendations are requested than there are viable parameter configurations

baybe/parameters/numerical.py

@@ -136,7 +136,7 @@ def is_in_range(self, item: float) -> bool:
 
     @override
     @property
-    def comp_rep_columns(self) -> tuple[str, ...]:
+    def comp_rep_columns(self) -> tuple[str]:
         return (self.name,)
 
     @override
@@ -150,5 +150,38 @@ def summary(self) -> dict:
         return param_dict
 
 
+@define(frozen=True, slots=False)
+class _FixedNumericalContinuousParameter(ContinuousParameter):
+    """Parameter class for fixed numerical parameters."""
+
+    is_numeric: ClassVar[bool] = True
+    # See base class.
+
+    value: float = field(converter=float)
+    """The fixed value of the parameter."""
+
+    @property
+    def bounds(self) -> Interval:
+        """The value of the parameter as a degenerate interval."""
+        return Interval(self.value, self.value)
+
+    @override
+    def is_in_range(self, item: float) -> bool:
+        return item == self.value
+
+    @override
+    @property
+    def comp_rep_columns(self) -> tuple[str]:
+        return (self.name,)
+
+    @override
+    def summary(self) -> dict:
+        return dict(
+            Name=self.name,
+            Type=self.__class__.__name__,
+            Value=self.value,
+        )
+
+
 # Collect leftover original slotted classes processed by `attrs.define`
 gc.collect()