cyclic_boosting package#

Subpackages#

Submodules#

cyclic_boosting.GBSregression module#

Cyclic Boosting Regression for Generalized Background Subtraction regression.

class cyclic_boosting.GBSregression.CBGBSRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, minimal_loss_change=1e-10, minimal_factor_change=1e-10, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, regalpha=0.0, aggregate=True)[source]#

Bases: RegressorMixin, CyclicBoostingBase, IdentityLinkMixin

Variant form of Cyclic Boosting’s location regressor, that corresponds to the regression of the outcome of a previous statistical subtraction of two classes of observations from each other (e.g. groups A and B: A - B).

For this, the target y has to be set to positive values for group A and negative values for group B.

Additional Parameter#

regalpha: float: A hyperparameter to steer the strength of regularization, i.e. a shrinkage of the regression result for A _B to 0. A value of 0 corresponds to no regularization.

calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data) → Tuple[ndarray, ndarray][source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

loss(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors) → None[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBGBSRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

cyclic_boosting.base module#

class cyclic_boosting.base.CyclicBoostingBase(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: LinkFunction, BaseEstimator

The CyclicBoostingBase class implements the core skeleton for all Cyclic Boosting algorithms. It learns parameters (e.g., factors) for each feature-bin combination and applies smoothing algorithms (supplied by the smoother argument) on the parameters.

By virtue of the feature_group parameter, each cyclic_boosting estimator can consider one- or higher-dimensional feature combinations.

The parameter estimates are iteratively optimized until one of the stop criteria (min_loss_change, minimal_factor_change, maximal_iterations) is reached.

Parameters:

feature_groups (sequence of column labels) –
(str or int) or tuples of such labels or cyclic_boosting.base.FeatureID. For each feature or feature tuple in the sequence, a one- or multidimensional factor profile will be determined, respectively, and used in the prediction.

If this argument is omitted, all columns except a possible weight_column are considered as one-dimensional feature_groups.
hierarchical_feature_groups (sequence of column labels) –
(str or int) or tuples of such labels or cyclic_boosting.base.FeatureID. In the first three iterations of the training, only the feature groups defined here are used, i.e., all other feature groups are excluded. From the fourth iteration onwards, all feature groups are used. The idea of such hierarchical iterations is to support the modeling of hierarchical or causal effects (e.g., mitigate confounding).

If this argument is not explicitly set, no such hierarchical iterations are run.
feature_properties (dict of int) –
Dictionary listing the names of all features for the training as keys and their pre-processing flags as values. When using a numpy feature matrix X with no column names the keys of the feature properties are the column indices.

By default, flags.HAS_MISSING is assumed. If flags.MISSING_NOT_LEARNED is set for a feature group, the neutral factor 1 is used for all non-finite values.
weight_column (string or int) – Column name or index used to include a weight column presented in X. If weight_column is None (default) equal weights for all samples are used.
prior_prediction_column (string, int or None) – Column name or index used to include prior predictions. Instead of the target mean (prior_prediction_column=None), the predictions found in the prior_prediction_column column are used as a base model in fit() and predict().
minimal_loss_change (float) – Stop if the relative loss change of current and previous prediction falls below this value.
minimal_factor_change (float) – Stop if the mean of the factor change falls below this value. The mean of the factor change is the mean of all absolute factor differences between the current and the previous iteration for each factor.
maximal_iterations (int) – Maximal number of iteration.
observers (list) – list of observer objects from the observers module.
smoother_choice (subclass of cyclic_boosting.SmootherChoice) – Selects smoothers
output_column (string) – In case of the usage as a transformer, this column is added and filled with the predictions
learn_rate (Functor or None) – Functor that defines the learning rate of each cyclic boosting iteration. It has to satisfy the interface of type learning_rate.constant_learn_rate_one(). If None is specified, which is the default, the learning rate is always 1.
aggregate (boolean or None) – Description is required

Notes

Preconditions

Non-negative integer values starting at 0 without gaps are expected in each feature column.
The number of unique values in each feature column n_unique_feature should be much smaller than the number of samples n.

Features transformed by cyclic_boosting.binning.BinNumberTransformer satisfy these preconditions.

The target y has to be positive semi-definite.

global_scale_link_#

The global scale of the target y in the linked space.

Type:: float

stop_criteria_#

tuple of three stop criteria are satisfied: (stop_iterations, stop_factor_change, stop_loss_change)

Type:: tuple

abstract calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

calibrate_to_weighted_mean(feature: Feature) → ndarray[source]#

cb_features(X: ndarray, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data) → Tuple[int, Any, Any][source]#

feature_iteration(X: ndarray, y: ndarray, feature: Feature, pred: CBLinkPredictionsFactors, prefit_data)[source]#

fit(X: DataFrame | ndarray, y: ndarray | None = None) → LinkFunction | BaseEstimator[source]#

fit_transform(X: DataFrame, y: ndarray | None = None) → DataFrame[source]#

get_feature_contributions(X: DataFrame | ndarray) → Dict[str, ndarray][source]#

Returns the contributions of each feature for each individual prediction on a given data set, offering individual explainability of the model. That means the learned parameter, e.g., factor for multiplicative modes, of the corresponding feature bin is accessed. These contributions should be interpreted in respect to the neutral element of the mode, i.e., 1 for multiplicative and 0 for additive modes.

Parameters:: X (np.ndarray) – data set to be predicted, needs to include the features of the model
Returns:: contributions from each feature for each sample of X
Return type:: dict

get_feature_importances() → Dict[str, float][source]#: Returns the relative importance of each input feature.

get_state() → Dict[str, Any][source]#

get_subestimators_as_items(prototypes=True) → List[Tuple][source]#: For prototypes=False, the clones are indexed by the column name or index.

property global_scale_: ndarray#: Obtain the global scale in the space of the original target y.

inc_fitting = False#

is_diverged(loss) → bool[source]#

learning_rate(feature: Feature) → float[source]#

loss(prediction: ndarray, y: ndarray, weights: ndarray) → ndarray[source]#

no_deepcopy = {'feature_properties'}#

abstract precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

predict(X: DataFrame | ndarray, y: ndarray | None = None) → ndarray[source]#

predict_extended(X: DataFrame | ndarray, influence_categories: dict | None = None) → CBLinkPredictionsFactors[source]#

prepare_plots(X: ndarray, y: ndarray, prediction: ndarray) → None[source]#

remove_preds(pred: CBLinkPredictionsFactors, X: ndarray) → None[source]#

required_columns() → Set[source]#

set_feature_importances() → None[source]#

supports_pandas = True#

transform(X: DataFrame, y: ndarray | None = None) → DataFrame[source]#

visit_factors(feature: Feature, unfitted_factors, X: ndarray, y: ndarray, pred: CBLinkPredictionsFactors) → None[source]#

cyclic_boosting.base.calc_factors_generic(lex_binnumbers: ndarray, w_x: ndarray, w: ndarray, w_x2: ndarray, external_weights: ndarray, minlength: int, x0: float, w0: float) → Tuple[float, float][source]#

Generic calculation of factors and uncertainties

It is always possible to reparametrise $\chi^2$ of the different cyclic boosting models to the following standard form

\[\chi^2 =\sum_i w_i \cdot (x_i - \hat{x})^2\]

where the sum goes over the samples in one specific feature bin and $\hat{x}$ is the parameter to be estimated.

Multiplicative Ansatz:

Here we have

\[\chi^2=\sum_i v_i \cdot (y_i - \hat{x} \cdot p_i)^2 / \sigma^2_{y_i} =\sum_i v_i p_i^2 \cdot (y_i/p_i - \hat{x})^2 / \sigma^2_{y_i}\]

where $v_i$ is the external weight, $y_i$ is the target, $p_i$ is the prediction without the contribution of this feature and $\sigma^2_{y_i}$ is the variance of the target. By setting

\[x_i = y_i / p_i\]

\[w_i = v_i \cdot p_i^2 / \sigma^2_{y_i}\]

we get the standard form.

Sum Ansatz:

Here we have

\[\chi^2=\sum_i v_i \cdot (y_i - (\hat{x} - p_i))^2 / \sigma^2_{y_i} =\sum_i v_i \cdot ((y_i - p_i) - \hat{x})^2 / \sigma^2_{y_i}\]

By setting

\[x_i = y_i - p_i\]

\[w_i = v_i / \sigma^2_{y_i}\]

we get the standard form.

Slope Ansatz:

Here we have

\[\chi^2=\sum_i v_i \cdot (y_i - (\hat{x} \cdot u_i + p_i))^2 / \sigma^2_{y_i} =\sum_i v_i u_i^2 \cdot ((y_i - p_i)/u_i - \hat{x})^2 / \sigma^2_{y_i}\]

where $u_i$ is the external column, e.g. a price ratio for dynamic pricing models. By setting

\[x_i = (y_i- p_i) / u_i\]

\[w_i = v_i \cdot u_i^2 / \sigma^2_{y_i}\]

we get the standard form.

All models above fit in the general linear ansatz:

\[\chi^2=\sum_i v_i \cdot (y_i - (\hat{x} \cdot m_i + b_i))^2 / \sigma^2_{y_i} =\sum_i v_i m_i^2 \cdot ((y_i - b_i)/m_i - \hat{x})^2 / \sigma^2_{y_i}\]

By setting

\[x_i = (y_i - b_i) / m_i\]

\[w_i = v_i \cdot m_i^2 / \sigma^2_{y_i} = \frac{v_i}{\sigma^2_{y_i} \cdot (\frac{d x_i}{d y_i})^2}\]

we get the standard form.

Solution for the general linear ansatz:

If we solve for $\hat{x}$, we get the formula for the weighted mean:

\[\hat{x} = \frac{\sum_i w_i \cdot x_i} {\sum_i w_i}\]

The variance of $\hat{x}$ is

\[\sigma^2_{\hat{x}} = \sum_i \left(\frac{d \hat{x}}{d x_i}\right)^2 \cdot \sigma^2_{x_i} = \sum_i \left(\frac{w_i}{\sum_i w_i}\right)^2 \cdot \sigma^2_{x_i}\]

with

\[w_i \cdot \sigma^2_{x_i} = \frac{v_i}{\sigma^2_{y_i} \cdot (\frac{d x_i}{d y_i})^2} \cdot \sigma^2_{y_i} \cdot \left(\frac{d x_i}{d y_i}\right)^2 = v_i\]

we can write the variance as

\[\sigma_{\hat{x}} = \frac{\sum_i w_i \cdot v_i} {(\sum_i w_i)^2}\]

Solution for the general linear ansatz with a prior on: $\hat{x}$

To handle bins with low statistics or empty bins we introduce a gaussian prior with precision $w_0$ and mean $x_0$

Our $\chi^2$ now has the following form:

\[\chi^2 =\sum_i w_i \cdot (x_i - \hat{x})^2 + w_0 \cdot (\hat{x} - x_0)^2\]

Solving for $\hat{x}$ gives

\[\hat{x} = \frac{x_0 \cdot w_0 + \sum_i w_i \cdot x_i} {w_0 + \sum_i w_i}\]

The variance of $\hat{x}$ is

\[\sigma^2_{\hat{x}} = \left(\frac{d \hat{x}}{d x_0}\right)^2 \cdot \sigma^2_{x_0} + \sum_i \left(\frac{d \hat{x}}{d x_i}\right)^2 \cdot \sigma^2_{x_i} = \frac{w_0 + \sum_i w_i \cdot v_i} {(w_0 + \sum_i w_i)^2}\]

Note

The quadratic sum is split in summands because parts of it may not be finite (e.g. in slope estimation).

Parameters:

lex_binnumbers (numpy.ndarray) – 1-dimensional numpy array containing the bin numbers.
w_x (numpy.ndarray) – $w_i \cdot x_i$ for each array element i
w (numpy.ndarray) – $w_i$ for each array element i
w_x2 (numpy.ndarray) – $w_i \cdot x_i^2$ for each array element i
external_weights (numpy.ndarray) – $v_i$ for each array element i
minlength (int) – number of bins for this feature including the nan bin
x0 (float) – mean of the prior on the mean
w0 (float) – prior precision of the prior on the mean

Returns:

A tuple of factors and uncertainties in the original space.

Return type:

tuple

cyclic_boosting.base.gaussian_matching_by_quantiles(dist, link_func, perc1: float | ndarray, perc2: float | ndarray) → Tuple[float, float][source]#

Gaussian matching of the distribution transformed by the link function by demanding the equality of two quantiles in link space

Since each quantile of a distribution is transformed just by the link function itself (by contrast to the mean or variance of the distribution), the Gaussian matching is quite easy and general:

link_func(dist.ppf(perc<i>)) = gaussian_quantile<i>

The connection between a Gaussian quantile and the distribution parameters is:

\[\text{gaussian\_quantile}_i = \mu + \sigma \sqrt{2} \text{erf}^{-1}( 2 \cdot \text{perc}_i - 1)\]

Applied to both perc<i>, this leads to a system of two linear equations that can be solved for mu and sigma.

Parameters:

dist (scipy.stats distribution object) – the assumed distribution in the original space, e.g. a scipy.stats.beta in the case of CBClassifier; it is assumed that this distribution has been initialized with vectorized parameters with one value for each cyclic boosting bin
link_func (callable) – the transformation function to link space, e.g. the method link_func() inherited in the cyclic boosting class
perc1 (float or ndarray of floats) – percentage (a fixed value or an individual value for each of the bins) of the first quantile to match
perc2 (float or ndarray of floats) – percentage (a fixed value or an individual value for each of the bins) of the second quantile to match; it must be different from perc1.

Returns:

mu, sigma of the matched Gaussian, each being a ndarray with one value for each bin

Return type:

tuple

Example

As example, consider the case of a beta distribution with link function logit which is used in cyclic_boosting.classification. It compares the old and new Gaussian matching for different alphas and betas.

Derivation of the transformation of the beta distribution to logit space:

\[ \begin{align}\begin{aligned}y = \text{logit}(x) = \log(x / (1 - x))\\\Leftrightarrow x = \frac{1}{1 + e^{-x}}\\\Rightarrow \frac{\mathrm{d} x}{\mathrm{d} y} = \frac{e^{-y}}{\left(1 + e^{-y}\right)^2}\\p_{\text{beta}'(\alpha, \beta)}(y)\,\mathrm{d} y = p_{\text{beta}(\alpha, \beta)}(x)\,\mathrm{d} x = p_{\text{beta}(\alpha, \beta)}(x(y)) \,\frac{\mathrm{d} x}{\mathrm{d} y}\,\mathrm{d} y\\= p_{\text{beta}(\alpha, \beta)}\left(\left(1 + e^{-y}\right)^2\right) \frac{e^{-y}}{\left(1 + e^{-y}\right)^2}\,\mathrm{d} y\end{aligned}\end{align} \]

cyclic_boosting.classification module#

Cyclic Boosting Classifier

class cyclic_boosting.classification.CBClassifier(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: ClassifierMixin, CyclicBoostingBase, LogitLinkMixin

This regressor is the cyclic boosting core algorithm for classifications

Its interface, methods and arguments are described in CyclicBoostingBase.

calc_parameters(feature: Feature, y: ndarray, pred, prefit_data: ndarray) → Tuple[float, float][source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature: Feature, y: ndarray, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

predict(X: DataFrame | ndarray, y: ndarray | None = None) → ndarray[source]#

predict_proba(X: DataFrame | ndarray, y: ndarray | None = None) → ndarray[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBClassifier#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

cyclic_boosting.classification.boost_weights(y: ndarray, prediction: ndarray) → ndarray[source]#

Returns weights for bincount operations on the CBClassifier.

The weights are assigned so that they are suitable for boosting, i.e. weights for well-estimated samples are low, weights for bad estimations are high.

\[\begin{split}w_i = \begin{cases} (1 - \left\langle y \right\rangle_i) & y_{\text{truth}} = 1 \\ \left\langle y \right\rangle_i & \text{otherwise} \end{cases}\end{split}\]

cyclic_boosting.classification.get_beta_priors() → Tuple[float, float][source]#

Prior values for beta distribution. The prior distribution was chosen to be Beta(1.001, 1.001), which is almost a uniform distribution but has a probability density function that goes to 0 for math:x=0 and $x=1$.

Returns:: $alpha=1.001$ and $beta=1.001$
Return type:: float

cyclic_boosting.common_smoothers module#

Convenient smoother mappings from feature property values (and tuples) to typical smoothers used for the Cyclic Boosting regressor

class cyclic_boosting.common_smoothers.SmootherChoice(use_regression_type=True, use_normalization=True, explicit_smoothers=None)[source]#

Bases: object

Base class for selecting smoothers for cyclic boosting.

Maps feature property tuples to smoothers in 1D/2D/3D.

Parameters:

use_regression_type (bool) – Flag to decide if RegressionType regularization should be used for the smoothers. (default = True)
use_normalization (bool) – Flag to decide if normalization should be used for the smoothers. (default = True)
explicit_smoothers (dict) – A dictionary with custom 1-d smoothers that override the default one-dimensional smoothers chosen by the feautre property. Needs to be of the format {feature_group : smoother}, where feature_group is a tuple of strings and smoother is and instance of AbstractBinSmoother. (default = None)

choice_fct(feature_group, feature_property, feature_type=None)[source]#

Returns the smoother specified by the get_raw_smoother method If an explicit smoother is defined for the feature group, the explicit smoother is used instead.

The result is wrapped with a meta_smoother using the wrap_smoother method.

Parameters:

feature_group (tuple) – Name of the feature_group
feature_property (tuple) – Tuple of feature properties.
feature_type – Type of the feature.

get_onedim_smoother(feature_property, feature_name=None)[source]#

Returns the standard one-dimensional smoother to be used for a specific feature. If an explicit 1D-smoother for the feature is defined, it is returned, otherwise the default smoother for the feature property is chosen.

Parameters:

feature_property (int) – Feature property defined as flag.
feature_name (str) – Name of the feature

get_raw_smoother(feature_group, feature_property, feature_type=None)[source]#

Method returning the raw smoother for the feature_group, feature_property and feature_type specified.

This is smoother is not yet wrapped with a meta_smoother from the wrap_smoother method.

Parameters:

feature_group (tuple) – Name of the feature_group
feature_property (tuple) – Tuple of feature properties.
feature_type – Type of the feature.

neutral_factor_link = 0#

wrap_smoother(smoother, feature_group, feature_property, feature_type)[source]#

Wrapper method that chooses the correct meta_estimators for the smoothers in the SmootherChoice class.

Parameters:

smoother (AbstractBinSmoother) – smoother that should be wrapped.
feature_group (tuple) – Name of the feature_group
feature_property (tuple) – Tuple of feature properties.
feature_type – Type of the feature.

class cyclic_boosting.common_smoothers.SmootherChoiceGroupBy(use_regression_type=True, use_normalization=True, explicit_smoothers=None)[source]#

Bases: SmootherChoice

Groupby smoothing for multi-dimensional feature groups.

get_raw_smoother(feature_group, feature_prop, feature_type=None)[source]#

Method returning the raw smoother for the feature_group, feature_property and feature_type specified.

This is smoother is not yet wrapped with a meta_smoother from the wrap_smoother method.

Parameters:

feature_group (tuple) – Name of the feature_group
feature_property (tuple) – Tuple of feature properties.
feature_type – Type of the feature.

wrap_smoother(smoother, feature_group, feature_property, feature_type=None)[source]#

Wrapper method that chooses the correct meta_estimators for the smoothers in the SmootherChoice class.

Parameters:

smoother (AbstractBinSmoother) – smoother that should be wrapped.
feature_group (tuple) – Name of the feature_group
feature_property (tuple) – Tuple of feature properties.
feature_type – Type of the feature.

class cyclic_boosting.common_smoothers.SmootherChoiceWeightedMean(use_regression_type=True, use_normalization=True, explicit_smoothers=None)[source]#

Bases: SmootherChoice

Weighted mean smoothing for multi-dimensional feature groups.

This defines a set of common smoothers where choose_smoothers_for_factor_model selects from.

get_raw_smoother(feature_group, feature_prop, feature_type=None)[source]#

Method returning the raw smoother for the feature_group, feature_property and feature_type specified.

This is smoother is not yet wrapped with a meta_smoother from the wrap_smoother method.

Parameters:

feature_group (tuple) – Name of the feature_group
feature_property (tuple) – Tuple of feature properties.
feature_type – Type of the feature.

cyclic_boosting.features module#

class cyclic_boosting.features.Feature(feature_id: FeatureID, feature_property: tuple | int, smoother: AbstractBinSmoother, minimal_factor_change: float | None = 0.0001)[source]#

Bases: object

Wrapper for information regarding a single feature group in the cyclic boosting algorithm

Parameters:

feature_id (FeatureID) – feature ID is a named tuple consisting of the name of a feature or tuple of such names in the multidimensional case and the feature type.
feature_property (int) – feature property, see flags
smoother (subclass of cyclic_boosting.smoothing.base.AbstractBinSmoother) – smoother for the bins of the feature group
minimal_factor_change (float) – minimum change of the results in the feature group’s bins that are considered progress in a cyclic boosting iteration

Note

The suffix _link in attributes refers to values valid in link space.

property bin_centers: ndarray#

bind_data(X: DataFrame | ndarray, weights: ndarray) → None[source]#

Binds data from X belonging to the feature and calculates the following features:

lex_binned_data: The binned data transformed to a 1-dimensional array
containing lexical binned data.
n_multi_bins_finite: Number of bins for each column in feature
(needed for plotting).
finite_bin_weightsums: Array containing the sum of weights for
each bin.
nan_bin_weightsum: The sum of weights for the nan-bin.
bin_centers: Array containing the center of each bin.

clear_feature_reference(observers: List) → None[source]#

property dim: int#: Dimension of the feature group, i.e. the number of its features

property factor_link_nan_bin: ndarray#

property finite_bin_weightsums: ndarray#: Array of finite bin weightsums

property is_1dim: bool#: True if feature is 1-dimensional, False otherwise

property missing_not_learned: bool#: True if flags.MISSING_NOT_LEARNED can be found in feature property.

property n_bins: int#: Number of bins, including the extra bin for missing and infinite values

property n_bins_finite: int#: Number of bins, excluding the extra bin for missing and infinite values

property nan_bin_weightsum: ndarray#

prepare_feature() → None[source]#

set_feature_bin_deviations_from_neutral(neutral_factor_link: float) → None[source]#

unbind_data() → None[source]#: Clear some of the references set in bind_data().

unbind_factor_data() → None[source]#

property unfitted_factor_link_nan_bin: ndarray#

update_factors(unfitted_factors: ndarray, unfitted_uncertainties: ndarray, neutral_factor_link: float, learn_rate: float) → ndarray[source]#

Call the smoother on the bin results if necessary.

Parameters:

unfitted_factors (ndarray) – bin means in link space as returned by the method cyclic_boosting.base.CyclicBoostingBase.calc_parameters()
unfitted_uncertainties (ndarray) – bin uncertainties in link space as returned by the method cyclic_boosting.base.CyclicBoostingBase.calc_parameters()
neutral_factor_link (float) – neutral value in link space, currently always 0
learn_rate (float) – learning rate used

class cyclic_boosting.features.FeatureID(feature_group, feature_type)#

Bases: tuple

feature_group#: Alias for field number 0

feature_type#: Alias for field number 1

class cyclic_boosting.features.FeatureList(features: List[Feature])[source]#

Bases: object

Iterable providing the information about a list of features in the form of Feature objects

Parameters:: features (list of Feature) – List of Feature that is normally created by create_features().

property feature_groups: List[tuple | str | int]#: Obtain the feature_groups for all features.

property feature_ids: List[FeatureID]#: Obtain the feature_groups for all features.

get_feature(feature_group: str | int | tuple, feature_type: str | None = None) → Feature[source]#

Selects feature specified by feature_group and feature_type

Parameters:

feature_group (string, int or tuple of string or int) – name or index of feature group
feature_type (None or string) – type of feature

Returns:

class – Feature instance

Return type:

cyclic_boosting.base.Feature

iter_fitting()[source]#: Generator yielding only the features with attribute stop_iterations == False

class cyclic_boosting.features.FeatureTypes[source]#

Bases: object

external = 'external'#

standard = None#

cyclic_boosting.features.create_feature_id(feature_group_or_id: FeatureID | Any, default_type: str | None = None) → FeatureID[source]#: Convenience function to convert feature_groups into :class:`FeatureID`s

cyclic_boosting.features.create_features(feature_groups_or_ids: List[str | int | tuple | FeatureID], feature_properties: dict, smoother_choice: SmootherChoice) → FeatureList[source]#

Parameters:

feature_groups_or_ids (List of string, int or tuple of string or FeatureID) – feature identifier
feature_properties (dict of feature properties) –
smoother_choice (subclass of cyclic_boosting.SmootherChoice) – Selects smoothers

cyclic_boosting.flags module#

cyclic_boosting.flags.check_flags_consistency(feature_prop: int) → None[source]#

Check that exactly one of IS_CONTINUOUS, IS_ORDERED, IS_UNORDERED has been set.

Parameters:: feature_prop (int) – value to check for consistency

Examples

>>> from cyclic_boosting.flags import check_flags_consistency
>>> from cyclic_boosting import flags

The following flags will just work:

>>> check_flags_consistency(flags.IS_CONTINUOUS)
>>> check_flags_consistency(flags.IS_ORDERED)
>>> check_flags_consistency(flags.IS_UNORDERED)

Flags that fail:

>>> check_flags_consistency(flags.HAS_MISSING)
Traceback (most recent call last):
ValueError: Exactly one of IS_CONTINUOUS, IS_ORDERED, IS_UNORDERED ...

>>> check_flags_consistency(flags.IS_CONTINUOUS | flags.IS_UNORDERED)
Traceback (most recent call last):
ValueError: Exactly one of IS_CONTINUOUS, IS_ORDERED, IS_UNORDERED ...

>>> check_flags_consistency(flags.IS_CONTINUOUS | flags.IS_ORDERED)
Traceback (most recent call last):
ValueError: Exactly one of IS_CONTINUOUS, IS_ORDERED, IS_UNORDERED ...

>>> check_flags_consistency(flags.IS_ORDERED | flags.IS_UNORDERED)
Traceback (most recent call last):
ValueError: Exactly one of IS_CONTINUOUS, IS_ORDERED, IS_UNORDERED ...

>>> check_flags_consistency(flags.DECREASING | flags.INCREASING)
Traceback (most recent call last):
ValueError: One feature can either be ...

cyclic_boosting.flags.decreasing_set(feature_prop: int) → bool[source]#

Is DECREASING included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import _set
>>> from cyclic_boosting import flags
>>> decreasing_set(flags.DECREASING)
True
>>> decreasing_set(flags.DECREASING | flags.HAS_MISSING)
True
>>> decreasing_set(flags.IS_MONOTONIC)
False

cyclic_boosting.flags.flags_to_string(flags_value: int | tuple) → tuple | str[source]#

This function converts the numeric flags to the corresponding strings that are defined in the flag list.

Parameters:: flags_value (int/tuple) – preprocessing flag (see cyclic_boosting.flags)
Returns:: Flag value
Return type:: str

Examples

>>> from cyclic_boosting.flags import flags_to_string
>>> from cyclic_boosting import flags
>>> flags_to_string(flags.IS_CONTINUOUS)
'IS_CONTINUOUS'

>>> flags_to_string(flags.IS_UNORDERED)
'IS_UNORDERED'

>>> flags_to_string(flags.IS_ORDERED)
'IS_ORDERED'

>>> flags_to_string(flags.IS_MONOTONIC)
'IS_CONTINUOUS | IS_MONOTONIC'

>>> flags_to_string(flags.HAS_MISSING | flags.IS_CONTINUOUS)
'IS_CONTINUOUS | HAS_MISSING'

>>> flags_to_string(flags.IS_UNORDERED | flags.HAS_MISSING
... | flags.MISSING_NOT_LEARNED)
'IS_UNORDERED | HAS_MISSING | MISSING_NOT_LEARNED'

>>> flags_to_string((flags.IS_ORDERED, flags.HAS_MISSING
... | flags.IS_CONTINUOUS))
('IS_ORDERED', 'IS_CONTINUOUS | HAS_MISSING')

>>> flags_to_string(flags.IS_CONTINUOUS | flags.IS_UNORDERED)
'IS_CONTINUOUS | IS_UNORDERED'

cyclic_boosting.flags.has_magic_missing_set(feature_prop: int) → bool[source]#

cyclic_boosting.flags.has_missing_set(feature_prop: int) → bool[source]#

Is HAS_MISSING included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import has_missing_set
>>> from cyclic_boosting import flags
>>> has_missing_set(flags.HAS_MISSING)
True
>>> has_missing_set(flags.IS_CONTINUOUS | flags.HAS_MISSING)
True
>>> has_missing_set(flags.IS_CONTINUOUS)
False

MISSING_NOT_LEARNED implies HAS_MISSING:

>>> has_missing_set(flags.MISSING_NOT_LEARNED)
True

cyclic_boosting.flags.increasing_set(feature_prop: int) → bool[source]#

Is INCREASING included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import _set
>>> from cyclic_boosting import flags
>>> _increasing_set(flags.INCREASING)
True
>>> increasing_set(flags.INCREASING | flags.HAS_MISSING)
True
>>> increasing_set(flags.IS_MONOTONIC)
False

cyclic_boosting.flags.is_continuous_set(feature_prop: int) → bool[source]#

Is IS_CONTINUOUS included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import is_continuous_set
>>> from cyclic_boosting import flags
>>> is_continuous_set(flags.IS_CONTINUOUS)
True
>>> is_continuous_set(flags.IS_CONTINUOUS | flags.HAS_MISSING)
True
>>> is_continuous_set(flags.IS_ORDERED)
False

IS_MONOTONIC implies IS_CONTINUOUS:

>>> is_continuous_set(flags.IS_MONOTONIC)
True

cyclic_boosting.flags.is_monotonic_set(feature_prop: int) → bool[source]#

Is IS_MONOTONIC included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import is_monotonic_set
>>> from cyclic_boosting import flags
>>> is_monotonic_set(flags.IS_MONOTONIC)
True
>>> is_monotonic_set(flags.IS_MONOTONIC | flags.HAS_MISSING)
True
>>> is_monotonic_set(flags.IS_CONTINUOUS)
False
>>> is_monotonic_set(flags.IS_CONTINUOUS | flags.IS_MONOTONIC)
True
>>> is_monotonic_set(flags.IS_ORDERED)
False
>>> is_monotonic_set(flags.IS_MONOTONIC_INCREASING)
True

cyclic_boosting.flags.is_ordered_set(feature_prop: int) → bool[source]#

Is IS_ORDERED included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import is_ordered_set
>>> from cyclic_boosting import flags
>>> is_ordered_set(flags.IS_ORDERED)
True
>>> is_ordered_set(flags.IS_ORDERED | flags.HAS_MISSING)
True
>>> is_ordered_set(flags.IS_CONTINUOUS)
False

cyclic_boosting.flags.is_unordered_set(feature_prop: int) → bool[source]#

Is IS_UNORDERED included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import is_unordered_set
>>> from cyclic_boosting import flags
>>> is_unordered_set(flags.IS_UNORDERED)
True
>>> is_unordered_set(flags.IS_UNORDERED | flags.HAS_MISSING)
True
>>> is_unordered_set(flags.IS_ORDERED)
False
>>> is_unordered_set(flags.IS_CONTINUOUS)
False

cyclic_boosting.flags.missing_not_learned_set(feature_prop: int) → bool[source]#

Is MISSING_NOT_LEARNED included in the feature properties?

Parameters:: feature_prop (int) – feature properties for one feature
Return type:: bool

>>> from cyclic_boosting.flags import missing_not_learned_set
>>> from cyclic_boosting import flags
>>> missing_not_learned_set(flags.MISSING_NOT_LEARNED)
True
>>> missing_not_learned_set(flags.IS_CONTINUOUS | flags.MISSING_NOT_LEARNED)
True
>>> missing_not_learned_set(flags.HAS_MISSING)
False
>>> missing_not_learned_set(flags.IS_CONTINUOUS)
False

cyclic_boosting.flags.read_feature_property(feature_properties: dict, feature_group: tuple | str, default: int) → tuple | int[source]#

Read a feature property out of the feature_properties dict which may be None. If no value is found, return default as the default value.

Parameters:

feature_properties (dict) – the feature_properties dict
feature_group (str or tuple of str) – feature group name
default (int) – the default value to return if feature_properties is None or doesn’t contain feature_group as a key.

>>> from cyclic_boosting.flags import read_feature_property, flags_to_string
>>> from cyclic_boosting import flags
>>> feature_properties = {'a': flags.IS_ORDERED, 'b': flags.IS_UNORDERED}
>>> flags_to_string(read_feature_property(feature_properties, 'b', flags.HAS_MISSING))
'IS_UNORDERED'
>>> flags_to_string(read_feature_property(feature_properties, 'c', flags.HAS_MISSING))
'HAS_MISSING'

cyclic_boosting.learning_rate module#

cyclic_boosting.learning_rate.constant_learn_rate_one(iteration: int, maximal_iteration: int, feature: Feature | None = None) → float[source]#

Function to specify the learning rate of a cyclic boosting iteration. The learning rate returned is always 1.

Parameters:

iteration (int) – Current major cyclic boosting iteration.
maximal_iteration (int) – Maximal cyclic boosting iteration.
feature (cyclic_boosting.base.Feature) – Feature

cyclic_boosting.learning_rate.half_linear_learn_rate(iteration: int, maximal_iteration: int, feature: Feature | None = None) → float[source]#

Function to specify the learning rate of a cyclic boosting iteration. The learning rate is linear increasing each iteration until it reaches 1 in half of the iterations.

Parameters:

iteration (int) – Current major cyclic boosting iteration.
maximal_iteration (int) – Maximal cyclic boosting iteration.
feature (cyclic_boosting.base.Feature) – Feature

cyclic_boosting.learning_rate.linear_learn_rate(iteration: int, maximal_iteration: int | float, feature: Feature | None = None) → float[source]#

Function to specify the learning rate of a cyclic boosting iteration. The learning rate is linear increasing each iteration until it reaches 1 in the last iteration.

Parameters:

iteration (int) – Current major cyclic boosting iteration.
maximal_iteration (int) – Maximal cyclic boosting iteration.
feature (cyclic_boosting.base.Feature) – Feature

cyclic_boosting.learning_rate.logistic_learn_rate(iteration: int, maximal_iteration: int, feature: Feature | None = None) → float[source]#

Function to specify the learning rate of a cyclic boosting iteration. The learning rate has a logistic form.

Parameters:

iteration (int) – Current major cyclic boosting iteration.
maximal_iteration (int) – Maximal cyclic boosting iteration.
feature (cyclic_boosting.base.Feature) – Feature

cyclic_boosting.link module#

This module contains some general/canonical link-mean-function pairs such as

LogLinkMixin
LogitLinkMixin

class cyclic_boosting.link.IdentityLinkMixin[source]#

Bases: LinkFunction

Identity link

is_in_range(m: ndarray)[source]#: Check if values can be transformed by the link function.

link_func(m: ndarray)[source]#: Returns a copy of the input

unlink_func(l: ndarray)[source]#: Returns a copy of the input

class cyclic_boosting.link.LinkFunction[source]#

Bases: object

Abstract base class for link function computations.

abstract is_in_range(values: ndarray)[source]#: Check if values can be transformed by the link function.

abstract link_func(m: ndarray)[source]#: Transform values in m to link space

abstract unlink_func(l: ndarray)[source]#: Inverse of link_func()

class cyclic_boosting.link.LogLinkMixin[source]#

Bases: LinkFunction

Link function and mean function for example for Poisson-distributed data.

Supported values are in the range $x > 0$

is_in_range(m: ndarray) → bool[source]#: Check if values can be transformed by the link function.

link_func(m: ndarray) → ndarray[source]#: Calculates the log-link

\[l = \log(\mu)\]

unlink_func(l: ndarray) → ndarray[source]#: Calculates the inverse of the link function

\[\mu = \exp(l)\]

class cyclic_boosting.link.LogitLinkMixin[source]#

Bases: LinkFunction

Link for the logit transformation.

Supported values are in the range $0 \leq x \leq 1$

is_in_range(p: ndarray) → bool[source]#: Check if values can be transformed by the link function.

link_func(p: ndarray) → ndarray[source]#: Calculates the logit-link

\[l = \log(\frac{p}{1-p})\]

unlink_func(l: ndarray) → ndarray[source]#: Inverse of logit-link

\[p = \frac{1}{1+ \exp(-l)}\]

cyclic_boosting.location module#

Cyclic Boosting Regression for a location parameter target.

class cyclic_boosting.location.CBLocPoissonRegressor(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: CyclicBoostingBase, RegressorMixin, IdentityLinkMixin

calc_parameters(feature, y, pred, prefit_data=None)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature, y, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

predict(X, y=None)[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBLocPoissonRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.location.CBLocationRegressor(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: RegressorMixin, CyclicBoostingBase, IdentityLinkMixin

calc_parameters(feature, y, pred, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
prediction_link (np.ndarray) – prediction in link space of all other features.
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

calibrate_to_weighted_mean(feature)[source]#

precalc_parameters(feature, y, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
prediction_link (np.ndarray) – prediction in link space.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBLocationRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

cyclic_boosting.nbinom module#

Cyclic Boosting Negative Binomial c regressor. var = mu + c * mu * mu

class cyclic_boosting.nbinom.CBNBinomC(mean_prediction_column, feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=0.0001, minimal_factor_change=0.0001, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=<function constant_learn_rate_one>, gamma=0.0, bayes=False, n_steps=15)[source]#

Bases: CyclicBoostingBase, RegressorMixin, LogitLinkMixin

Maximum Likelihood estimation of the c parameter of the Negative Binomial Negative Binomial Variance: mu + c * mu**2 Estimator predicts c with the constraint: c in (0,1) Follows https://en.wikipedia.org/wiki/Negative_binomial_distribution notation with c = 1/r

Parameters:

mean_prediction_column (string or None) – Column for the mean of the Negative Binomial
gamma (float) – Lasso term, zero for non-penalized fit. The larger the value the harder the regularization.
bayes (bool) – use expectation of the posterior instead of maximum likelihood in each cyclic boosting step
CyclicBoostingBase. (The rest of the parameters are documented in) –

calc_parameters(feature, y, pred, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

fit(X, y=None)[source]#

loss(c, y, weights)[source]#

precalc_parameters(feature, y, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBNBinomC#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

cyclic_boosting.nbinom.bayes_result(loss: Array(float64, 2, 'A', False, aligned=True), minlength: int64, new_c_link: Array(float64, 1, 'A', False, aligned=True))[source]#

cyclic_boosting.nbinom.binned_loss_nbinom_c(y: Array(float64, 1, 'A', False, aligned=True), mu: Array(float64, 1, 'A', False, aligned=True), c_link: Array(float64, 1, 'A', False, aligned=True), binnumbers: Array(int64, 1, 'A', False, aligned=True), minlength: int64, gamma: float64, new_c_link: float64)[source]#

cyclic_boosting.nbinom.calc_parameters_nbinom_c(y, mu, c_link, binnumbers, minlength, gamma, bayes, new_c_link)[source]#

cyclic_boosting.nbinom.compute_2d_loss(y: Array(float64, 1, 'A', False, aligned=True), mu: Array(float64, 1, 'A', False, aligned=True), c_link: Array(float64, 1, 'A', False, aligned=True), binnumbers: Array(int64, 1, 'A', False, aligned=True), minlength: int64, gamma: float64, new_c_link: Array(float64, 1, 'A', False, aligned=True))[source]#

cyclic_boosting.nbinom.get_new_c_link_for_iteration(iteration, n_steps)[source]#

cyclic_boosting.nbinom.loss_nbinom_c(y: Array(float64, 1, 'A', False, aligned=True), mu: Array(float64, 1, 'A', False, aligned=True), c: Array(float64, 1, 'A', False, aligned=True), gamma: float64) → float64[source]#

cyclic_boosting.nbinom.nbinom_log_pmf(x: float64, n: float64, p: float64) → float64[source]#: Negative binomial log PMF.

cyclic_boosting.observers module#

class cyclic_boosting.observers.BaseObserver[source]#

Bases: object

Base class for observers

Observers are used to extract information from cyclic boosting estimators that might be of further interest, but are not needed by the estimator in order to make predictions.

observe_feature_iterations(iteration, feature_i, X, y, prediction, weights, estimator_state)[source]#

Called after each feature has been processed as part of a full iteration of the algorithm.

Parameters:

iteration (int) – number of the iteration
feature_i (int) – number of the feature that was processed
X (pandas.DataFrame or numpy.ndarray) – feature matrix
y (numpy.ndarray) – target array
prediction (numpy.ndarray) – current target prediction
weights (numpy.ndarray) – target weights
estimator_state (dict) – state of the estimator. See get_state() for information on what will be passed here
local_variables (dict) – local variables

observe_iterations(iteration, X, y, prediction, weights, estimator_state)[source]#

Called after each iteration of the algorithm.

Parameters:

iteration (int) – number of the iteration
X (pandas.DataFrame or numpy.ndarray) – feature matrix
y (numpy.ndarray) – target array
prediction (numpy.ndarray) – current target prediction
weights (numpy.ndarray) – target weights
estimator_state (dict) – state of the estimator. See cyclic_boosting.base.CyclicBoostingBase.get_state() for information on what will be passed here

class cyclic_boosting.observers.PlottingObserver(iteration=-1)[source]#

Bases: BaseObserver

Observer retrieving all information necessary to obtain analysis plots based on a cyclic boosting training.

Instances of this class are intended to be passed as elements of the observers parameter to a cyclic boosting estimator, where each will gather information on a specific iteration. Afterwards, they can be passed to plot_analysis().

Parameters:: iteration (int) – The observer will save all necessary information for the analysis plots based on the state of the internal variables of the estimator after the given iteration has been calculated. Default is -1, which signifies the last iteration.

check_fitted()[source]#

observe_feature_iterations(iteration, feature_i, X, y, prediction, weights, estimator_state)[source]#

Observe iterations in cyclic_boosting estimator to collect information for necessary for plots. This function is called in each feature/minor loop.

Parameters:

iteration (int) – current major iteration number of cyclic boosting loop
feature_i (int) – current minor iteration number of cyclic boosting loop
X (pd.DataFrame or numpy.ndarray shape(n, k)) – feature matrix
y (np.ndarray) – target array
prediction (np.ndarray) – array of current cyclic boosting prediction
weights (np.ndarray) – array of event weights
estimator_state (dict) – state object of cyclic_boosting estimator

observe_iterations(iteration, X, y, prediction, weights, estimator_state, delta=None, quantile=None)[source]#

Observe iterations in cyclic_boosting estimator to collect information for necessary for plots. This function is called in each major loop and once in the end.

Parameters:

iteration (int) – current major iteration of cyclic boosting loop
X (pd.DataFrame or numpy.ndarray shape(n, k)) – feature matrix
y (np.ndarray) – target array
prediction (np.ndarray) – array of current cyclic boosting prediction
weights (np.ndarray) – array of event weights
estimator_state (dict) – state object of cyclic_boosting estimator

cyclic_boosting.observers.calc_in_sample_histograms(y, pred, weights, quantile=None)[source]#

Calculates histograms for use with diagonal plot.

Parameters:

y (numpy.ndarray) – truth
pred (numpy.ndarray) – prediction
weights (np.ndarray) – array of event weights

Returns:

result – Tuple consisting of:

means
bin_centers
errors
counts

Return type:

tuple

cyclic_boosting.pipelines module#

cyclic_boosting.pipelines.pipeline_CB(estimator=None, feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=0.001, minimal_factor_change=0.001, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, number_of_bins=100, aggregate=True, a=1.0, c=0.0, external_colname=None, standard_feature_groups=None, external_feature_groups=None, var_prior_exponent=0.1, prior_exponent_colname=None, mean_prediction_column=None, gamma=0.0, bayes=False, n_steps=15, regalpha=0.0, quantile=None, costs=None, inplace=False)[source]#

cyclic_boosting.pipelines.pipeline_CBAdditiveGenericCRegressor(**kwargs)[source]#: Convenience function containing CBAdditiveGenericCRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBAdditiveQuantileRegressor(**kwargs)[source]#: Convenience function containing CBAdditiveQuantileRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBClassifier(**kwargs)[source]#: Convenience function containing CBClassifier (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBExponential(**kwargs)[source]#: Convenience function containing CBExponential (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBGBSRegressor(**kwargs)[source]#: Convenience function containing CBGBSRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBGenericClassifier(**kwargs)[source]#: Convenience function containing CBGenericClassifier (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBLocPoissonRegressor(**kwargs)[source]#: Convenience function containing CBLocPoissonRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBLocationRegressor(**kwargs)[source]#: Convenience function containing CBLocationRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBMultiplicativeGenericCRegressor(**kwargs)[source]#: Convenience function containing CBMultiplicativeGenericCRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBMultiplicativeQuantileRegressor(**kwargs)[source]#: Convenience function containing CBMultiplicativeQuantileRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBNBinomC(**kwargs)[source]#: Convenience function containing CBNBinomC (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBNBinomRegressor(**kwargs)[source]#: Convenience function containing CBNBinomRegressor (estimator) + binning.

cyclic_boosting.pipelines.pipeline_CBPoissonRegressor(**kwargs)[source]#: Convenience function containing CBPoissonRegressor (estimator) + binning.

cyclic_boosting.price module#

Cyclic boosting exponential price model estimator

class cyclic_boosting.price.CBExponential(external_colname, standard_feature_groups, external_feature_groups, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=0.001, minimal_factor_change=0.001, maximal_iterations=20, observers=None, var_prior_exponent=0.1, smoother_choice=None, prior_exponent_colname=None, output_column=None, learn_rate=None, a=1.0, c=0.0)[source]#

Bases: CBNBinomRegressor

calc_parameters(feature, y, pred, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

Parameters:

feature (Feature) – feature information
y (np.ndarray) – target, truth
prediction_link (np.ndarray) – prediction in link space of all other features.
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

fit(X, y=None)[source]#

predict(X, y=None)[source]#

predict_extended(X, influence_categories)[source]#

required_columns()[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBExponential#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.price.CBLinkPredictions(predictions, exponents, base)[source]#

Bases: UpdateMixin

Support for prediction of type log(p) = factors + base * exponents

base()[source]#

exponents()[source]#

factors()[source]#

predict_link()[source]#

prior_exponents()[source]#

cyclic_boosting.price.combine_lists_of_feature_groups(standard_feature_groups, external_feature_groups)[source]#: Combine the feature_groups and external_feature_groups to one list of feature_ids.

cyclic_boosting.price.gamma_momemt_matching(factors, variance_factors, link_func)[source]#

cyclic_boosting.price.newton_bisect(newton_step, args, valid, x_l, x_r, epsilon=0.001, maximal_iterations=10, start_values=None)[source]#

Combination of Newton’s Method and Bisection to find the root of the jacobian.

Every time Newton tries a step outside the bounding interval [x_l, x_r] the bisection update is used.

Parameters:

newton_step (callable) –

Calculate the parameter update, jacobian and hessian:

def newton_step(l, *args):
    ...
    return l_new, jacobian, hessian

args (iterable) – Parameters pass to newton_step
valid (np.ndarray) – Boolean array encoding valid values
x_l (np.ndarray) – Start values for the left bounds
x_r (np.ndarray) – Start values for the right bounds
epsilon (float) –
Epsilon on the jacobian. Iteration stops if:
```
np.all(np.abs(jacobian) < epsilon)
```
maximal_iterations (int) – number of maximal iterations
start_values (np.ndarray or None) – Start values on the parameters. If None, the middle of the first bisection interval is used.

Returns:

(fitted parameters, variance of parameters,: left bounds, right bounds)

Return type:

tuple

cyclic_boosting.price.newton_step(l, y, p, log_x, k, prior, s, log_k_prior, var_l, lex_binnumbers, minlength, only_jac=False)[source]#

cyclic_boosting.regression module#

class cyclic_boosting.regression.CBNBinomRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=0.001, minimal_factor_change=0.001, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, a=1.0, c=0.0, aggregate=True)[source]#

Bases: CBBaseRegressor

This regressor minimizes the mean squared error. It is usable for regressions of target-values $0 \leq y < \infty$.

This Cyclic Boosting mode assumes an underlying negative binomial (or Gamma-Poisson) distribution as conditional distribution of the target. The prior values for the Gamma distribution $\alpha = 2.0$ and $\beta = 1.67834$ are chosen such that its median is $\Gamma_{\text{median}}(\alpha, \beta) =1$, which is the neutral element of multiplication (Cyclic Boosting in this mode is a multiplicative model). The estimate for each factor is the median of the Gamma distribution with the measured values of $\alpha$ and $\beta$. To determine the uncertainties of the factors, the variance is estimated from a log-normal distribution that is approximated using the first two moments of the Gamma distribution.

In the default case of parameter settings $a = 1.0$ and $c = 0.0$, this regressor corresponds to the special case of a Poisson regressor, as implemented in CBPoissonRegressor.

calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data) → Tuple[ndarray][source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBNBinomRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.regression.CBPoissonRegressor(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: CBBaseRegressor

This regressor minimizes the mean squared error. It is usable for regressions of target-values $0 \leq y < \infty$.

As Poisson regressor, it is a special case of the more general negative binomial regressor CBNBinomRegressor, assuming purely Poisson-distributed target values.

calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBPoissonRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

cyclic_boosting.regression.get_gamma_priors() → Tuple[int, float][source]#: prior values for Gamma distribution with median 1

cyclic_boosting.utils module#

exception cyclic_boosting.utils.ConvergenceError[source]#

Bases: RuntimeError

Exception type, that should be thrown if an estimator cannot converge and therefore its result cannot be trusted.

class cyclic_boosting.utils.ConvergenceParameters(loss_change: float = 1e+20, delta: float = 100.0)[source]#

Bases: object

Class for registering the convergence parameters

delta: float = 100.0#

loss_change: float = 1e+20#

set_delta(updated_delta: float) → None[source]#

set_loss_change(updated_loss_change: float) → None[source]#

cyclic_boosting.utils.arange_multi(stops) → ndarray[source]#

Multidimensional generalization of numpy.arange()

Parameters:: stops (sequence of numbers) – upper limits (exclusive) for the corresponding dimensions in the result
Returns:: matrix of combinations of range values in lexicographic order
Return type:: numpy.ndarray (element-type int64, shape (n_bins, len(stops)))

>>> from cyclic_boosting.utils import arange_multi
>>> arange_multi([2, 3])
array([[0, 0],
       [0, 1],
       [0, 2],
       [1, 0],
       [1, 1],
       [1, 2]])

In this example, the last dimension has only one bin:

>>> arange_multi([3, 4, 1])
array([[0, 0, 0],
       [0, 1, 0],
       [0, 2, 0],
       [0, 3, 0],
       [1, 0, 0],
       [1, 1, 0],
       [1, 2, 0],
       [1, 3, 0],
       [2, 0, 0],
       [2, 1, 0],
       [2, 2, 0],
       [2, 3, 0]])

>>> arange_multi([2])
array([[0],
       [1]])

cyclic_boosting.utils.bin_steps(n_bins: Array(int64, 1, 'A', False, aligned=True))[source]#

Multidimensional bin steps for lexicographical order

Parameters:

n_bins (numpy.ndarray (element-type float64, shape (M,))) – number of bins for each dimension

Returns:

in slot i + 1 for dimension i, the number of steps needed for iterating through the following dimensions in lexicographical order until the bin number can be increased for dimension i.

See the doctests of arange_multi(): It’s the number of rows between changes in column i.

In slot 0 this is the number steps needed to iterate through all dimensions.

Return type:

numpy.ndarray (element-type int64, shape (M + 1,))

>>> from cyclic_boosting.utils import bin_steps
>>> bin_steps(np.array([3, 2]))
array([6, 2, 1])

>>> bin_steps(np.array([3, 2, 4, 1, 2]))
array([48, 16,  8,  2,  2,  1])

cyclic_boosting.utils.calc_linear_bins(x, nbins)[source]#

Calculate a linear binning for all values in x with nbins

Parameters:

x (list or numpy.ndarray (float64, dim=1)) – Input array.
nbins (int) – number of bins desired

Example

>>> x = np.array([0.0, 0.1, 0.3, 0.4, 0.8, 0.9, 0.95, 1.1, 1.3, 1.5])
>>> nbins = 3
>>> from cyclic_boosting.utils import calc_linear_bins
>>> bin_boundaries, bin_centers = calc_linear_bins(x, nbins)
>>> bin_centers
array([ 0.25,  0.75,  1.25])
>>> bin_boundaries
array([ 0. ,  0.5,  1. ,  1.5])

cyclic_boosting.utils.calc_means_medians(binnumbers, y, weights=None)[source]#

Calculate the means, medians, counts, and errors for y grouped over the binnumbers.

Parameters:

binnumbers (...) – binnumbers
y (list or numpy.ndarray (float64, dim=1)) – target values
weights (list or numpy.ndarray (float64, dim=1)) – array of event weights
**Example** –
np.array([0.0 (>>> y =) –
0. –
0. –
1. –
1. –
1. –
2. –
2. –
3. –
3.]) –
np.array([0.0 –
0.2 (0.0) –
0.5 –
0.6 –
0.7 (1.0) –
0.85 –
1.0 –
1.2 –
1.4 –
1.6]) –
calc_means_medians (>>> from cyclic_boosting.utils import) –
means (>>>) –
medians (>>>) –
counts (>>>) –
calc_means_medians( (errors =) –
binnumbers –
y) –
means –
0.233333 (0.0) –
0.716667 (1.0) –
1.100000 (2.0) –
1.500000 (3.0) –
dtype (int64) –
medians –
0.2 –
0.7 –
1.1 (2.0) –
1.5 (3.0) –
dtype –
errors (>>>) –
-0.13654 ([0.0) –
-0.06827 (3.0) –
-0.06827 –
-0.06827 –
dtype –
0.102405 (1.0) –
0.068270 (3.0) –
0.068270 –
dtype –
-0.09545 (3.0) –
-0.09545 –
-0.09545 –
dtype –
0.143175 (1.0) –
0.095450 (3.0) –
0.095450 –
dtype –
counts –
3 (1.0) –
3 –
2 (3.0) –
2 –
dtype –

cyclic_boosting.utils.calc_weighted_quantile(binnumbers, y, weights, quantile)[source]#

cyclic_boosting.utils.clone(estimator, safe=True)[source]#

Constructs a new estimator with the same constructor parameters.

A better name for this function would be ‘reconstruct’.

This is a reimplemenation of sklearn.base.clone() respecting wishes of estimators and their subestimators to avoid reconstructing certain attributes.

Clone performs a deep copy of the model in an estimator without actually copying attached data. It yields a new estimator with the same parameters that has not been fit on any data.

cyclic_boosting.utils.continuous_quantile_from_discrete_pdf(y, quantile, weights)[source]#

Calculates a continous quantile value approximation for a given quantile from an array of potentially discrete values.

Parameters:

y (np.ndarray) – containing data with float type (potentially discrete)
quantile (float) – desired quantile

Returns:

calculated quantile value

Return type:

float

cyclic_boosting.utils.digitize(x, bins)[source]#

This is an alternative version of numpy.digitize(). It puts x values greater or equal to bins.max() on the last index of bins. Values smaller than bins.min() are put on the first index of bins.

Parameters:

x (list or numpy.ndarray (float64, dim=1)) – Input array to be binned. It has to be 1-dimensional.
bins (list or numpy.ndarray (float64, dim=1)) – Array of bins. It has to be 1-dimensional and monotonic.

Returns:

Output array of indices, of same shape as x.

Return type:

list or numpy.ndarray (float64, dim=1)

Raises

ValueError: If the input is not 1-dimensional, or if bins is not monotonic.
TypeError: If the type of the input is complex.

See Also

numpy.digitize()

Notes

If values in x are such that they fall outside the bin range, attempting to index bins with the indices that digitize returns will result in an IndexError.

Examples

>>> x = np.array([-1000, -0.2, 0.2, 6.4, 3.0, 10, 11, 1000])
>>> bins = np.array([0.0, 1.0, 2.5, 4.0, 10.0])
>>> from cyclic_boosting.utils import digitize
>>> inds = digitize(x, bins)
>>> inds
array([0, 0, 1, 4, 3, 4, 4, 4])

cyclic_boosting.utils.generator_to_decorator(gen)[source]#

Turn a generator into a decorator.

The mechanism is similar to contextlib.contextmanager() which turns a generator into a contextmanager. decorator is used internally.

Thanks to decorator, this function preserves the docstring and the signature of the function to be decorated.

The docstring of the resulting decorator will include the original docstring of the generator and an additional remark stating that this is the corresponding decorator.

Parameters:: gen (generator function) – wrapping the function call.
Returns:: decorator

For a detailed example, see generator_to_decorator_and_contextmanager().

cyclic_boosting.utils.get_X_column(X, column, array_for_1_dim=True)[source]#

Picks columns from pandas.DataFrame or numpy.ndarray.

Parameters:

X (pandas.DataFrame or numpy.ndarray) – Data Source from which columns are picked.
column – The format depends on the type of X. For pandas.DataFrame you can give a string or a list/tuple of strings naming the columns. For numpy.ndarray an integer or a list/tuple of integers indexing the columns.
array_for_1_dim (bool) – In default mode (set to True) the return type for a one dimensional access is a np.ndarray with shape (n, ). If set to False it is a np.ndarray with shape (1, n).

cyclic_boosting.utils.get_bin_bounds(binners, feat_group)[source]#

Gets the bin boundaries for each feature group.

Parameters:

binners (list) – List of binners.
feat_group (str or tuple of str) – A feature property for which the bin boundaries should be extracted from the binners.

cyclic_boosting.utils.get_feature_column_names(X, exclude_columns=[])[source]#

cyclic_boosting.utils.get_normalized_values(values: Iterable) → List[float][source]#

cyclic_boosting.utils.linear_regression(x, y, w)[source]#

Performs a linear regression allowing uncertainties in y.

$f(x) = y = \alpha + \beta \cdot x$

Parameters:

x (numpy.ndarray) – x vector
y (numpy.ndarray) – y vector
w (numpy.ndarray) – weight vector

Returns:

The coefficients alpha and beta.

Return type:

tuple of 2 float

cyclic_boosting.utils.multidim_binnos_to_lexicographic_binnos(binnos, n_bins=None, binsteps=None)[source]#

Map index-tuples of M-dimensional features to integers.

In the cyclic boosting algorithm there is a one-dimensional array of factors for each one-dimensional feature. For processing of multi-dimensional variables (i.e. combinations of several one-dimensional feature variables) we have bins for all bin combinations. For example, if you have two one-dimensional features p and q with n_p and n_q bins, the two-dimensional feature (p, q) will have a two-dimensional factor array of shape (n_p, n_q).

The internal representation of this array, however, is that of a one-dimensional array. Thus, a function is needed that maps index-tuples (p, q,...) to integer indices. This is the purpose of this function.

This function performs this mapping for N rows of index M-tuples.

If there are any missing values in a row, the returned ordinal number is set to np.product(n_bins).

Parameters:

binnos (numpy.ndarray of shape (N, M)) – multidimensional bin numbers
n_bins (numpy.ndarray of shape (M,) or None) – number of bins in each dimension; if None, it will be determined from binnos.
binsteps (numpy.ndarray of type int64 and shape (M,)) – bin steps as returned by bin_steps() when called on n_bins; if None, it will be determined from binnos.

Returns:

ordinal numbers of the bins in lexicographic order as numpy.ndarray of type int64 and maximal bin numbers as numpy.ndarray of shape (M,)

Return type:

tuple

Examples

>>> binnos = np.c_[[1, 1, 0, 0, np.nan, 1, np.nan],
...                [0, 1, 2, 1, 1, 2, np.nan]]
>>> n_bins = np.array([2, 3])
>>> binsteps = bin_steps(n_bins)
>>> from cyclic_boosting.utils import multidim_binnos_to_lexicographic_binnos
>>> lex_binnos, n_finite = multidim_binnos_to_lexicographic_binnos(
...     binnos, n_bins=n_bins, binsteps=binsteps)
>>> lex_binnos
array([3, 4, 2, 1, 6, 5, 6])
>>> n_finite
array([2, 3])
>>> lex_binnos, n_finite =  multidim_binnos_to_lexicographic_binnos(binnos)
>>> lex_binnos
array([3, 4, 2, 1, 6, 5, 6])
>>> n_finite
array([2, 3])

cyclic_boosting.utils.nans(shape)[source]#

Create a new numpy array filled with NaNs.

Parameters:: shape (int or tuple) – shape of the numpy.ndarray
Return type:: numpy.ndarray (numpy.nan, dim=1)

>>> from cyclic_boosting.utils import nans
>>> nans(3)
array([ nan,  nan,  nan])

>>> nans((2, 2))
array([[ nan,  nan],
       [ nan,  nan]])

cyclic_boosting.utils.neutralize_one_dim_influence(values, uncertainties)[source]#

Neutralize influence of one dimensional features in a two-dimensional factor matrix

Parameters:

values (numpy.ndarray (float64, dim=2)) – measured values
uncertainties (numpy.ndarray (float64, dim=2)) – uncertainties for the values.

returns an updated 2D-table that has no net shift on any of the 2 projections

cyclic_boosting.utils.not_seen_events(x, wx, n_bins)[source]#

Return a boolean array that slices x so that only values with a non-finite weightsum wx are used.

Parameters:

x (pandas.DataFrame or numpy.ndarray) – Array can be of any dimension. The array has to consist out of contigous integers.
wx (numpy.ndarray weightsum of all bins occuring in x) –
n_bins (numpy.ndarray of shape (M,)) – number of bins in each dimension

Example

>>> x = np.c_[np.array([0, 1, 2, 3, 4, np.nan, -1])]
>>> wx = np.array([3, 2, 0, 1, 0])
>>> nbins = np.array([4, 1])
>>> from cyclic_boosting.utils import not_seen_events
>>> not_seen_events(x, wx, nbins)
array([False, False, True, False,  True,  True,  True], dtype=bool)

>>> x = pd.DataFrame({"a": [0, 1, 0, 1, np.nan, 0, -1],
...                   "b": [0, 0, 1, 1, 1, np.nan, -1]})
>>> wx = np.array([1, 0, 1, 0, 1])
>>> nbins = np.array([2, 2])
>>> not_seen_events(x, wx, nbins)
array([False, False, True, True, False, False, False], dtype=bool)

cyclic_boosting.utils.regularize_to_error_weighted_mean(values, uncertainties, prior_prediction=None)[source]#

Regularize values with uncertainties to its error-weighted mean.

Parameters:

values (numpy.ndarray (float64, dim=1)) – measured values
uncertainties (numpy.ndarray (float64, dim=1)) – uncertainties for the values.
prior_prediction (float) – If the prior_prediction is specified, all values are regularized with it and not with the error weighted mean.

Returns:

regularized values

Return type:

numpy.ndarray (float64, dim=1)

The error weighted mean is defined as:

\[\bar{x}_{\sigma} = \frac{\sum\limits_{i} \frac{1} {\sigma^2_i} \cdot x_i}{\sum\limits_{i} \frac{1}{\sigma^2_i}}\]

with the uncertainties $\sigma_i$ and values $x_i$. The uncertainty $\sigma_{\bar{x}}$ for $\bar{x}$ is calculated as:

\[\sigma^2_{\bar{x}} = \frac{\sum\limits_{i} \frac{1}{\sigma^2_i} (x - \bar{x}_{\sigma})^2}{\sum\limits_{i} \frac{1}{\sigma^2_i}}\]

The regularized values are calculated as follows:

\[x_i^{'} = \frac{\frac{1}{\sigma^2_i} \cdot x_i + \frac{1}{\sigma^2_{\bar{x}}} \cdot \bar{x}} {\frac{1}{\sigma^2_i} + \frac{1}{\sigma^2_{\bar{x}}}}\]

>>> values = np.array([100., 100., 100., 90., 110., 180., 180., 20., 20.])
>>> uncertainties = np.array([10., 10., 10., 10., 10., 10., 15., 10., 15.])
>>> from cyclic_boosting.utils import regularize_to_error_weighted_mean
>>> regularize_to_error_weighted_mean(values, uncertainties)
array([ 100.        ,  100.        ,  100.        ,   90.40501997,
        109.59498003,  176.75984027,  173.06094747,   23.24015973,
         26.93905253])

>>> values = np.array([100.])
>>> uncertainties = np.array([10.])
>>> np.allclose(regularize_to_error_weighted_mean(values, uncertainties),
...             values)
True

>>> values = np.array([100., 101.])
>>> uncertainties = np.array([10., 10.])
>>> regularize_to_error_weighted_mean(
...     values, uncertainties, prior_prediction=50.)
array([ 98.11356348,  99.07583475])

>>> values = np.array([100., 10])
>>> uncertainties = np.array([10.])
>>> regularize_to_error_weighted_mean(values, uncertainties)
Traceback (most recent call last):
ValueError: <values> and <uncertainties> must have the same shape

cyclic_boosting.utils.regularize_to_error_weighted_mean_neighbors(values: ndarray, uncertainties: ndarray, window_size: int | None = 3) → ndarray[source]#

Regularize values with uncertainties to its error-weighted mean, using a sliding window.

Parameters:

values (np.ndarray) – data (float type) to be regularized
uncertainties (np.ndarray) – uncertainties (float type) of values
window_size (int) – size of the sliding window to be used (e.g., 3 means include direct left and right neighbors)

Returns:

regularized values

Return type:

np.ndarray

cyclic_boosting.utils.regularize_to_prior_expectation(values, uncertainties, prior_expectation, threshold=2.5)[source]#

Regularize values with uncertainties to a prior expectation.

Parameters:

values (numpy.ndarray (float64, dim=1)) – measured values
uncertainties (numpy.ndarray (float64, dim=1)) – uncertainties for the values.
prior_expectation (numpy.ndarray (float64, dim=1) or float) – The prior expectations dominate the regularized value if the uncertainties are large.
threshold (float) –
Threshold in terms of sigma. If the significance of a value:

\[\text{sig}(x_i) = \frac{x_i - \text{prior\_expectation}_i} {\text{uncertainty}_i}\]

is below the threshold, the prior expectation replaces the value.

Doctests

>>> values = np.array([0, 1, 2, 3])
>>> uncertainties = np.ones_like(values)*0.1
>>> prior_expectation = 1.
>>> from cyclic_boosting.utils import regularize_to_prior_expectation
>>> regularized_values = regularize_to_prior_expectation(
... values, uncertainties, prior_expectation)
>>> regularized_values
array([ 0.03175416,  1.        ,  1.96824584,  2.98431348])
>>> np.allclose(1 - np.sqrt(((values[0] - 1) / 0.1)**2 - 2.5**2) * 0.1,
...     regularized_values[0])
True
>>> np.allclose(1 + np.sqrt(((values[-1] - 1) / 0.1)**2 - 2.5**2) * 0.1,
...     regularized_values[-1])
True

cyclic_boosting.utils.slice_finite_semi_positive(x)[source]#

Return slice of all finite and semi positive definite values of x

Parameters:: x (pandas.DataFrame or numpy.ndarray) – Array can be of any dimension.

Example

>>> x = np.array([1, np.nan, 3, -2])
>>> from cyclic_boosting.utils import slice_finite_semi_positive
>>> slice_finite_semi_positive(x)
array([ True, False,  True, False], dtype=bool)

>>> X = pd.DataFrame({'a': [1, np.nan, 3], 'b': [-1, 2, 3]})
>>> slice_finite_semi_positive(X)
array([False, False,  True], dtype=bool)

cyclic_boosting.utils.smear_discrete_cdftruth(cdf_func: callable, y: int) → float[source]#

Smearing of the CDF value of a sample from a discrete random variable. Main usage is for a histogram of CDF values to check an estimated individual probability distribution (should be flat).

Parameters:

y (int) – value from discrete random variable
cdf_func (callable) – cumulative distribution function

Returns:

smeared CDF value for y

Return type:

float

cyclic_boosting.utils.smear_discrete_cdftruth_qpd(qpds: ndarray, y: ndarray) → ndarray[source]#

Smearing of the CDF values of samples from discrete random variables. Main usage is for a histogram of CDF values to check an estimated individual probability distribution (should be flat).

Parameters:

y (np.ndarray) – values from discrete random variables
qpds (np.ndarray) – array of QPDs

Returns:

smeared CDF values for y

Return type:

np.ndarray

cyclic_boosting.utils.weighted_stddev(values, weights)[source]#

Calculates the weighted standard deviation $\sigma$:

\[\sigma = \frac{\sum_i w_i \cdot (x_i - /mu_{wx})^2}{\sum_i w_i}\]

Parameters:

values (numpy.ndarray (float64, dim=1)) – Values of the samples $x_i$.
weights (numpy.ndarray (float64, dim=1)) – Weights of the samples $w_i$.

Module contents#

This package contains the Cyclic Boosting family of machine learning algorithms.

If you are looking for conceptional explanations of the Cyclic Boosting algorithm, you might have a look at the two papers https://arxiv.org/abs/2002.03425 and https://arxiv.org/abs/2009.07052.

API reference of the different Cyclic Boosting methods:

Multiplicative Regression

CBPoissonRegressor
CBNBinomRegressor
CBExponential
CBMultiplicativeQuantileRegressor
CBMultiplicativeGenericCRegressor

Additive Regression

CBLocationRegressor
CBLocPoissonRegressor
CBAdditiveQuantileRegressor
CBAdditiveGenericCRegressor

PDF Prediction

CBNBinomC

Classification

CBClassifier
CBGenericClassifier

Background Subtraction

CBGBSRegressor

class cyclic_boosting.CBAdditiveGenericCRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=1e-10, minimal_factor_change=1e-10, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, aggregate=True, costs=None)[source]#

Bases: CBGenericLoss, RegressorMixin, IdentityLinkMixin

Additive regression mode allowing an arbitrary loss function to be minimized in each feature bin. This should be used for unconstrained target ranges, i.e., -infinity to infinity.

Parameters:: costs (function) – loss (to be exact, cost) function to be minimized

:param See cyclic_boosting.base for all other parameters.:

costs(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

loss(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

model(param: float, yhat_others: ndarray) → ndarray[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBAdditiveGenericCRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

uncertainty(y: ndarray, weights: ndarray) → float[source]#

Estimation of parameter uncertainty for a given feature bin.

Parameters:

y (np.ndarray) – target variable, containing data with float type (potentially discrete)
weights (np.ndarray) – optional (otherwise set to 1) sample weights, containing data with float type

Returns:

estimated parameter uncertainty

Return type:

float

class cyclic_boosting.CBAdditiveQuantileRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=1e-10, minimal_factor_change=1e-10, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, quantile=None, aggregate=True)[source]#

Bases: CBQuantileRegressor, IdentityLinkMixin

Cyclic Boosting additive quantile-regression mode. A quantile loss, according to the desired quantile to be predicted, is minimized in each bin of each feature.

This should be used for unconstrained target ranges, i.e., -infinity to infinity.

Parameters:

quantile (float) – quantile to be estimated
See (class:cyclic_boosting.base for all other parameters.) –

model(param: float, yhat_others: ndarray) → ndarray[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBAdditiveQuantileRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

uncertainty(y: ndarray, weights: ndarray) → float[source]#

Estimation of parameter uncertainty for a given feature bin.

Parameters:

y (np.ndarray) – target variable, containing data with float type (potentially discrete)
weights (np.ndarray) – optional (otherwise set to 1) sample weights, containing data with float type

Returns:

estimated parameter uncertainty

Return type:

float

class cyclic_boosting.CBClassifier(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: ClassifierMixin, CyclicBoostingBase, LogitLinkMixin

This regressor is the cyclic boosting core algorithm for classifications

Its interface, methods and arguments are described in CyclicBoostingBase.

calc_parameters(feature: Feature, y: ndarray, pred, prefit_data: ndarray) → Tuple[float, float][source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature: Feature, y: ndarray, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

predict(X: DataFrame | ndarray, y: ndarray | None = None) → ndarray[source]#

predict_proba(X: DataFrame | ndarray, y: ndarray | None = None) → ndarray[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBClassifier#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CBExponential(external_colname, standard_feature_groups, external_feature_groups, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=0.001, minimal_factor_change=0.001, maximal_iterations=20, observers=None, var_prior_exponent=0.1, smoother_choice=None, prior_exponent_colname=None, output_column=None, learn_rate=None, a=1.0, c=0.0)[source]#

Bases: CBNBinomRegressor

calc_parameters(feature, y, pred, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

Parameters:

feature (Feature) – feature information
y (np.ndarray) – target, truth
prediction_link (np.ndarray) – prediction in link space of all other features.
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

fit(X, y=None)[source]#

predict(X, y=None)[source]#

predict_extended(X, influence_categories)[source]#

required_columns()[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBExponential#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CBGBSRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, minimal_loss_change=1e-10, minimal_factor_change=1e-10, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, regalpha=0.0, aggregate=True)[source]#

Bases: RegressorMixin, CyclicBoostingBase, IdentityLinkMixin

For this, the target y has to be set to positive values for group A and negative values for group B.

Additional Parameter#

regalpha: float: A hyperparameter to steer the strength of regularization, i.e. a shrinkage of the regression result for A _B to 0. A value of 0 corresponds to no regularization.

calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data) → Tuple[ndarray, ndarray][source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

loss(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors) → None[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBGBSRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CBGenericClassifier(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=1e-10, minimal_factor_change=1e-10, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, aggregate=True, costs=None)[source]#

Bases: CBGenericLoss, ClassifierMixin, LogitLinkMixin

Multiplicative (binary, i.e., target values 0 and 1) classification mode allowing an arbitrary loss function to be minimized in each feature bin.

Parameters:: costs (function) – loss (to be exact, cost) function to be minimized

:param See cyclic_boosting.base for all other parameters.:

costs(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

loss(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

model(param: float, yhat_others: ndarray) → ndarray[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBGenericClassifier#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

uncertainty(y: ndarray, weights: ndarray) → float[source]#

Estimation of parameter uncertainty for a given feature bin.

Parameters:

y (np.ndarray) – target variable, containing data with float type (potentially discrete)
weights (np.ndarray) – optional (otherwise set to 1) sample weights, containing data with float type

Returns:

estimated parameter uncertainty

Return type:

float

class cyclic_boosting.CBLocPoissonRegressor(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: CyclicBoostingBase, RegressorMixin, IdentityLinkMixin

calc_parameters(feature, y, pred, prefit_data=None)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature, y, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

predict(X, y=None)[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBLocPoissonRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CBLocationRegressor(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: RegressorMixin, CyclicBoostingBase, IdentityLinkMixin

calc_parameters(feature, y, pred, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
prediction_link (np.ndarray) – prediction in link space of all other features.
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

calibrate_to_weighted_mean(feature)[source]#

precalc_parameters(feature, y, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
prediction_link (np.ndarray) – prediction in link space.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBLocationRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CBMultiplicativeGenericCRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=1e-10, minimal_factor_change=1e-10, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, aggregate=True, costs=None)[source]#

Bases: CBGenericLoss, RegressorMixin, LogLinkMixin

Multiplicative regression mode allowing an arbitrary loss function to be minimized in each feature bin.This should be used for non-negative target ranges, i.e., 0 to infinity.

Parameters:: costs (function) – loss (to be exact, cost) function to be minimized

:param See cyclic_boosting.base for all other parameters.:

costs(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

loss(prediction: ndarray, y: ndarray, weights: ndarray) → float[source]#

model(param: float, yhat_others: ndarray) → ndarray[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBMultiplicativeGenericCRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

uncertainty(y: ndarray, weights: ndarray) → float[source]#

Estimation of parameter uncertainty for a given feature bin.

Parameters:

y (np.ndarray) – target variable, containing data with float type (potentially discrete)
weights (np.ndarray) – optional (otherwise set to 1) sample weights, containing data with float type

Returns:

estimated parameter uncertainty

Return type:

float

class cyclic_boosting.CBMultiplicativeQuantileRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=1e-10, minimal_factor_change=1e-10, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, quantile=None, aggregate=True)[source]#

Bases: CBQuantileRegressor, LogLinkMixin

Cyclic Boosting multiplicative quantile-regression mode. A quantile loss, according to the desired quantile to be predicted, is minimized in each bin of each feature. While its general structure allows arbitrary/empirical target ranges/distributions, the multiplicative model of this mode requires non-negative target values.

This should be used for non-negative target ranges, i.e., 0 to infinity.

Parameters:: quantile (float) – quantile to be estimated

:param See cyclic_boosting.base for all other parameters.:

model(param: float, yhat_others: ndarray) → ndarray[source]#

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBMultiplicativeQuantileRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

uncertainty(y: ndarray, weights: ndarray) → float[source]#

Estimation of parameter uncertainty for a given feature bin.

Parameters:

y (np.ndarray) – target variable, containing data with float type (potentially discrete)
weights (np.ndarray) – optional (otherwise set to 1) sample weights, containing data with float type

Returns:

estimated parameter uncertainty

Return type:

float

class cyclic_boosting.CBNBinomC(mean_prediction_column, feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=0.0001, minimal_factor_change=0.0001, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=<function constant_learn_rate_one>, gamma=0.0, bayes=False, n_steps=15)[source]#

Bases: CyclicBoostingBase, RegressorMixin, LogitLinkMixin

Parameters:

mean_prediction_column (string or None) – Column for the mean of the Negative Binomial
gamma (float) – Lasso term, zero for non-penalized fit. The larger the value the harder the regularization.
bayes (bool) – use expectation of the posterior instead of maximum likelihood in each cyclic boosting step
CyclicBoostingBase. (The rest of the parameters are documented in) –

calc_parameters(feature, y, pred, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

fit(X, y=None)[source]#

loss(c, y, weights)[source]#

precalc_parameters(feature, y, pred)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBNBinomC#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CBNBinomRegressor(feature_groups=None, hierarchical_feature_groups=None, feature_properties=None, weight_column=None, prior_prediction_column=None, minimal_loss_change=0.001, minimal_factor_change=0.001, maximal_iterations=10, observers=None, smoother_choice=None, output_column=None, learn_rate=None, a=1.0, c=0.0, aggregate=True)[source]#

Bases: CBBaseRegressor

This regressor minimizes the mean squared error. It is usable for regressions of target-values $0 \leq y < \infty$.

In the default case of parameter settings $a = 1.0$ and $c = 0.0$, this regressor corresponds to the special case of a Poisson regressor, as implemented in CBPoissonRegressor.

calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data) → Tuple[ndarray][source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBNBinomRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CBPoissonRegressor(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: CBBaseRegressor

This regressor minimizes the mean squared error. It is usable for regressions of target-values $0 \leq y < \infty$.

As Poisson regressor, it is a special case of the more general negative binomial regressor CBNBinomRegressor, assuming purely Poisson-distributed target values.

calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CBPoissonRegressor#

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:: sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.
Returns:: self – The updated object.
Return type:: object

class cyclic_boosting.CyclicBoostingBase(feature_groups=None, hierarchical_feature_groups=None, training_iterations_hierarchical_features=3, feature_properties: Dict[int, int] | None = None, weight_column: str | int | None = None, prior_prediction_column: str | int | None = None, minimal_loss_change: float | None = 1e-06, minimal_factor_change: float | None = 0.0001, maximal_iterations: int | None = 10, observers: list | None = None, smoother_choice: SmootherChoice | None = None, output_column: str | None = None, learn_rate: float | None = None, aggregate: bool | None = True)[source]#

Bases: LinkFunction, BaseEstimator

By virtue of the feature_group parameter, each cyclic_boosting estimator can consider one- or higher-dimensional feature combinations.

The parameter estimates are iteratively optimized until one of the stop criteria (min_loss_change, minimal_factor_change, maximal_iterations) is reached.

Parameters:

feature_groups (sequence of column labels) –
(str or int) or tuples of such labels or cyclic_boosting.base.FeatureID. For each feature or feature tuple in the sequence, a one- or multidimensional factor profile will be determined, respectively, and used in the prediction.

If this argument is omitted, all columns except a possible weight_column are considered as one-dimensional feature_groups.
hierarchical_feature_groups (sequence of column labels) –
(str or int) or tuples of such labels or cyclic_boosting.base.FeatureID. In the first three iterations of the training, only the feature groups defined here are used, i.e., all other feature groups are excluded. From the fourth iteration onwards, all feature groups are used. The idea of such hierarchical iterations is to support the modeling of hierarchical or causal effects (e.g., mitigate confounding).

If this argument is not explicitly set, no such hierarchical iterations are run.
feature_properties (dict of int) –
Dictionary listing the names of all features for the training as keys and their pre-processing flags as values. When using a numpy feature matrix X with no column names the keys of the feature properties are the column indices.

By default, flags.HAS_MISSING is assumed. If flags.MISSING_NOT_LEARNED is set for a feature group, the neutral factor 1 is used for all non-finite values.
weight_column (string or int) – Column name or index used to include a weight column presented in X. If weight_column is None (default) equal weights for all samples are used.
prior_prediction_column (string, int or None) – Column name or index used to include prior predictions. Instead of the target mean (prior_prediction_column=None), the predictions found in the prior_prediction_column column are used as a base model in fit() and predict().
minimal_loss_change (float) – Stop if the relative loss change of current and previous prediction falls below this value.
minimal_factor_change (float) – Stop if the mean of the factor change falls below this value. The mean of the factor change is the mean of all absolute factor differences between the current and the previous iteration for each factor.
maximal_iterations (int) – Maximal number of iteration.
observers (list) – list of observer objects from the observers module.
smoother_choice (subclass of cyclic_boosting.SmootherChoice) – Selects smoothers
output_column (string) – In case of the usage as a transformer, this column is added and filled with the predictions
learn_rate (Functor or None) – Functor that defines the learning rate of each cyclic boosting iteration. It has to satisfy the interface of type learning_rate.constant_learn_rate_one(). If None is specified, which is the default, the learning rate is always 1.
aggregate (boolean or None) – Description is required

Notes

Preconditions

Non-negative integer values starting at 0 without gaps are expected in each feature column.
The number of unique values in each feature column n_unique_feature should be much smaller than the number of samples n.

Features transformed by cyclic_boosting.binning.BinNumberTransformer satisfy these preconditions.

The target y has to be positive semi-definite.

global_scale_link_#

The global scale of the target y in the linked space.

Type:: float

stop_criteria_#

tuple of three stop criteria are satisfied: (stop_iterations, stop_factor_change, stop_loss_change)

Type:: tuple

abstract calc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data)[source]#

Calculates factors and uncertainties of the bins of a feature group in the original space (not the link space) and transforms them to the link space afterwards

The factors and uncertainties cannot be determined in link space, not least because target values like 0 diverge in link spaces like log or logit.

The main loop will call the smoothers on the results returned by this method. Smoothing is always done in link space.

This function must be implemented by the subclass according to its statistical model.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)
prefit_data – data returned by precalc_parameters() during fit

Returns:

This method must return a tuple of factors and uncertainties in the link space.

Return type:

tuple

calibrate_to_weighted_mean(feature: Feature) → ndarray[source]#

cb_features(X: ndarray, y: ndarray, pred: CBLinkPredictionsFactors, prefit_data) → Tuple[int, Any, Any][source]#

feature_iteration(X: ndarray, y: ndarray, feature: Feature, pred: CBLinkPredictionsFactors, prefit_data)[source]#

fit(X: DataFrame | ndarray, y: ndarray | None = None) → LinkFunction | BaseEstimator[source]#

fit_transform(X: DataFrame, y: ndarray | None = None) → DataFrame[source]#

get_feature_contributions(X: DataFrame | ndarray) → Dict[str, ndarray][source]#

Parameters:: X (np.ndarray) – data set to be predicted, needs to include the features of the model
Returns:: contributions from each feature for each sample of X
Return type:: dict

get_feature_importances() → Dict[str, float][source]#: Returns the relative importance of each input feature.

get_state() → Dict[str, Any][source]#

get_subestimators_as_items(prototypes=True) → List[Tuple][source]#: For prototypes=False, the clones are indexed by the column name or index.

property global_scale_: ndarray#: Obtain the global scale in the space of the original target y.

inc_fitting = False#

is_diverged(loss) → bool[source]#

learning_rate(feature: Feature) → float[source]#

loss(prediction: ndarray, y: ndarray, weights: ndarray) → ndarray[source]#

no_deepcopy = {'feature_properties'}#

abstract precalc_parameters(feature: Feature, y: ndarray, pred: CBLinkPredictionsFactors)[source]#

Calculations that are not dependent on intermediate predictions. If these are not needed, return None in the subclass.

Results returned by this method will be served to factors_and_uncertainties() as the prefit_data argument.

Parameters:

feature (Feature) – class containing all features
y (np.ndarray) – target, truth
pred – (in-sample) predictions from all other features (excluding the one at hand)

predict(X: DataFrame | ndarray, y: ndarray | None = None) → ndarray[source]#

predict_extended(X: DataFrame | ndarray, influence_categories: dict | None = None) → CBLinkPredictionsFactors[source]#

prepare_plots(X: ndarray, y: ndarray, prediction: ndarray) → None[source]#

remove_preds(pred: CBLinkPredictionsFactors, X: ndarray) → None[source]#

required_columns() → Set[source]#

set_feature_importances() → None[source]#

supports_pandas = True#

transform(X: DataFrame, y: ndarray | None = None) → DataFrame[source]#

visit_factors(feature: Feature, unfitted_factors, X: ndarray, y: ndarray, pred: CBLinkPredictionsFactors) → None[source]#

cyclic_boosting.pipeline_CBAdditiveGenericCRegressor(**kwargs)[source]#: Convenience function containing CBAdditiveGenericCRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBAdditiveQuantileRegressor(**kwargs)[source]#: Convenience function containing CBAdditiveQuantileRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBClassifier(**kwargs)[source]#: Convenience function containing CBClassifier (estimator) + binning.

cyclic_boosting.pipeline_CBExponential(**kwargs)[source]#: Convenience function containing CBExponential (estimator) + binning.

cyclic_boosting.pipeline_CBGBSRegressor(**kwargs)[source]#: Convenience function containing CBGBSRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBGenericClassifier(**kwargs)[source]#: Convenience function containing CBGenericClassifier (estimator) + binning.

cyclic_boosting.pipeline_CBLocPoissonRegressor(**kwargs)[source]#: Convenience function containing CBLocPoissonRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBLocationRegressor(**kwargs)[source]#: Convenience function containing CBLocationRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBMultiplicativeGenericCRegressor(**kwargs)[source]#: Convenience function containing CBMultiplicativeGenericCRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBMultiplicativeQuantileRegressor(**kwargs)[source]#: Convenience function containing CBMultiplicativeQuantileRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBNBinomC(**kwargs)[source]#: Convenience function containing CBNBinomC (estimator) + binning.

cyclic_boosting.pipeline_CBNBinomRegressor(**kwargs)[source]#: Convenience function containing CBNBinomRegressor (estimator) + binning.

cyclic_boosting.pipeline_CBPoissonRegressor(**kwargs)[source]#: Convenience function containing CBPoissonRegressor (estimator) + binning.