infomeasure.estimators.mutual_information package

Submodules

infomeasure.estimators.mutual_information.ansb module

Module for the Ansb mutual information estimator.

class infomeasure.estimators.mutual_information.ansb.AnsbCMIEstimator(*data, cond=None, undersampled: float = 0.1, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseAnsbMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Ansb mutual information.

Ansb conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

undersampledfloat, default=0.1

Maximum allowed ratio N/K to consider data sufficiently undersampled. A warning is issued if this threshold is exceeded.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.ansb.AnsbEntropyEstimator

Ansb entropy estimator.

Notes

This estimator uses the Ansb entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ansb.AnsbMIEstimator(*data, cond=None, undersampled: float = 0.1, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseAnsbMIEstimator, MutualInformationEstimator

Estimator for the Ansb mutual information.

Ansb mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

undersampledfloat, default=0.1

Maximum allowed ratio N/K to consider data sufficiently undersampled. A warning is issued if this threshold is exceeded.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.ansb.AnsbEntropyEstimator

Ansb entropy estimator.

Notes

This estimator uses the Ansb entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ansb.BaseAnsbMIEstimator(*data, cond=None, undersampled: float = 0.1, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Ansb mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

undersampledfloat, default=0.1

Maximum allowed ratio N/K to consider data sufficiently undersampled. A warning is issued if this threshold is exceeded.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

infomeasure.estimators.mutual_information.bayes module

Module for the Bayes mutual information estimator.

class infomeasure.estimators.mutual_information.bayes.BaseBayesMIEstimator(*data, cond=None, alpha: float | str, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Bayes mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

alphafloat | str

The concentration parameter α of the Dirichlet prior. Either a float or a string specifying the choice of concentration parameter.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.bayes.BayesCMIEstimator(*data, cond=None, alpha: float | str, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBayesMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Bayes mutual information.

Bayesian conditional mutual information estimator using Dirichlet prior with concentration parameter α. Provides principled handling of sparse data through Bayesian probability estimates.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

alphafloat | str

The concentration parameter α of the Dirichlet prior. Either a float or a string specifying the choice of concentration parameter.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bayes.BayesEntropyEstimator

Bayesian entropy estimator with Dirichlet prior.

Notes

This estimator uses Bayesian probability estimates with a Dirichlet prior to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.bayes.BayesMIEstimator(*data, cond=None, alpha: float | str, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBayesMIEstimator, MutualInformationEstimator

Estimator for the Bayes mutual information.

Bayesian mutual information estimator using Dirichlet prior with concentration parameter α. Provides principled handling of sparse data through Bayesian probability estimates.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

alphafloat | str

The concentration parameter α of the Dirichlet prior. Either a float or a string specifying the choice of concentration parameter.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bayes.BayesEntropyEstimator

Bayesian entropy estimator with Dirichlet prior.

Notes

This estimator uses Bayesian probability estimates with a Dirichlet prior to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

infomeasure.estimators.mutual_information.bonachela module

Module for the Bonachela mutual information estimator.

class infomeasure.estimators.mutual_information.bonachela.BaseBonachelaMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Bonachela mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.bonachela.BonachelaCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBonachelaMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Bonachela mutual information.

Bonachela conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bonachela.BonachelaEntropyEstimator

Bonachela entropy estimator.

Notes

This estimator uses the Bonachela entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.bonachela.BonachelaMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBonachelaMIEstimator, MutualInformationEstimator

Estimator for the Bonachela mutual information.

Bonachela mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bonachela.BonachelaEntropyEstimator

Bonachela entropy estimator.

Notes

This estimator uses the Bonachela entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

infomeasure.estimators.mutual_information.chao_shen module

Module for the ChaoShen mutual information estimator.

class infomeasure.estimators.mutual_information.chao_shen.BaseChaoShenMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for ChaoShen mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.chao_shen.ChaoShenCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoShenMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional ChaoShen mutual information.

ChaoShen conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_shen.ChaoShenEntropyEstimator

ChaoShen entropy estimator.

Notes

This estimator uses the ChaoShen entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.chao_shen.ChaoShenMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoShenMIEstimator, MutualInformationEstimator

Estimator for the ChaoShen mutual information.

ChaoShen mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_shen.ChaoShenEntropyEstimator

ChaoShen entropy estimator.

Notes

This estimator uses the ChaoShen entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

infomeasure.estimators.mutual_information.chao_wang_jost module

Module for the ChaoWangJost mutual information estimator.

class infomeasure.estimators.mutual_information.chao_wang_jost.BaseChaoWangJostMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for ChaoWangJost mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.chao_wang_jost.ChaoWangJostCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoWangJostMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional ChaoWangJost mutual information.

ChaoWangJost conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_wang_jost.ChaoWangJostEntropyEstimator

ChaoWangJost entropy estimator.

Notes

This estimator uses the ChaoWangJost entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.chao_wang_jost.ChaoWangJostMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoWangJostMIEstimator, MutualInformationEstimator

Estimator for the ChaoWangJost mutual information.

ChaoWangJost mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_wang_jost.ChaoWangJostEntropyEstimator

ChaoWangJost entropy estimator.

Notes

This estimator uses the ChaoWangJost entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

infomeasure.estimators.mutual_information.discrete module

Module for the discrete mutual information estimator.

class infomeasure.estimators.mutual_information.discrete.BaseDiscreteMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: DiscreteMIMixin, ABC

Base class for discrete mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like or tuple of array_like, optional

The conditional data used to estimate the conditional mutual information. Multiple RVs can be passed as tuples and will be treated as joint distributions.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.discrete.DiscreteCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseDiscreteMIEstimator, ConditionalMutualInformationEstimator

Estimator for the discrete conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

class infomeasure.estimators.mutual_information.discrete.DiscreteMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseDiscreteMIEstimator, MutualInformationEstimator

Estimator for the discrete mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

infomeasure.estimators.mutual_information.grassberger module

Module for the Grassberger mutual information estimator.

class infomeasure.estimators.mutual_information.grassberger.BaseGrassbergerMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Grassberger mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.grassberger.GrassbergerCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseGrassbergerMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Grassberger mutual information.

Grassberger conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.grassberger.GrassbergerEntropyEstimator

Grassberger entropy estimator.

Notes

This estimator uses the Grassberger entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.grassberger.GrassbergerMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseGrassbergerMIEstimator, MutualInformationEstimator

Estimator for the Grassberger mutual information.

Grassberger mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.grassberger.GrassbergerEntropyEstimator

Grassberger entropy estimator.

Notes

This estimator uses the Grassberger entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

infomeasure.estimators.mutual_information.kernel module

Module for the kernel-based mutual information estimator.

class infomeasure.estimators.mutual_information.kernel.BaseKernelMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: WorkersMixin, ABC

Base class for mutual information using Kernel Density Estimation (KDE).

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

workersint, optional

Number of workers to use for parallel processing. Default is 1, meaning no parallel processing. If set to -1, all available CPU cores will be used.

class infomeasure.estimators.mutual_information.kernel.KernelCMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKernelMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using Kernel Density Estimation (KDE).

\[I(X;Y|Z) = \sum_{i=1}^{n} p(x_i, y_i, z_i) \log \left( \frac{p(z_i)p(x_i, y_i, z_i)}{p(x_i, z_i)p(y_i, z_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

class infomeasure.estimators.mutual_information.kernel.KernelMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKernelMIEstimator, MutualInformationEstimator

Estimator for mutual information using Kernel Density Estimation (KDE).

\[I(X;Y) = \sum_{i=1}^{n} p(x_i, y_i) \log \left( \frac{p(x_i, y_i)}{p(x_i)p(y_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger module

Module for the Kraskov-Stoegbauer-Grassberger (KSG) mutual information estimator.

class infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger.BaseKSGMIEstimator(*data, cond=None, k: int = 4, ksg_id: int = 1, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

ksg_idint, default=1

The KSG estimator variant to use (1 or 2). Type I uses strict inequality for neighbor counting and the corresponding formula. Type II uses non-strict inequality and a different formula.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

class infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger.KSGCMIEstimator(*data, cond=None, k: int = 4, ksg_id: int = 1, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKSGMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

normalizebool, optional

If True, normalize the data before analysis.

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KSG11].

class infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger.KSGMIEstimator(*data, cond=None, k: int = 4, ksg_id: int = 1, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKSGMIEstimator, MutualInformationEstimator

Estimator for mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The estimator supports two variants:

• Type I (ksg_id=1): Uses strict inequality for counting neighbors in marginal spaces (dist < eps).

• Type II (ksg_id=2): Uses non-strict inequality (dist <= eps) and a modified formula.

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KSG11].

infomeasure.estimators.mutual_information.miller_madow module

Module for the discrete Miller-Madow mutual information estimator.

class infomeasure.estimators.mutual_information.miller_madow.BaseMillerMadowMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: DiscreteMIMixin, ABC

Base class for discrete Miller-Madow mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.miller_madow.MillerMadowCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseMillerMadowMIEstimator, ConditionalMutualInformationEstimator

Estimator for the discrete Miller-Madow conditional mutual information.

\[\begin{align} \hat{I}_{\tiny{MM}}(X_1; X_2; \ldots; X_n \mid Z)&= \hat{I}_{\tiny{MLE}}(\dots) + \frac{\left(\sum_{i=1}^{n}K_{iZ}-1\right) - \left(K_{1,\dots,i,Z}-1\right) - \left(K_{Z}-1\right)}{2N \cdot \log(\texttt{base})} \end{align}\]

\(\hat{I}_{\tiny{MLE}}(X_1; \dots; X_n \mid Z)\) is the initial DiscreteCMIEstimator estimate, \(K_{iZ}\) is the number of unique values in the i-th variable joint with Z, \(K_{1,\dots,i,Z}\) is the number of unique joint values, \(K_Z\) is the number of unique values in the Z variable, and \(N\) is the number of samples.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

class infomeasure.estimators.mutual_information.miller_madow.MillerMadowMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseMillerMadowMIEstimator, MutualInformationEstimator

Estimator for the discrete Miller-Madow mutual information.

\[\begin{split}\begin{align}\hat{I}_{\tiny{MM}}(X; Y) &= \hat{H}_{\tiny{MM}}(X) + \hat{H}_{\tiny{MM}}(Y) - \hat{H}_{\tiny{MM}}(X, Y)\\ &= \hat{H}_{\tiny{MLE}}(X) + \hat{H}_{\tiny{MLE}}(Y) - \hat{H}_{\tiny{MLE}}(X, Y) + (K_X + K_Y - K_{XY} - 1)/(2N \cdot \log(\texttt{base}))\\ &= \hat{I}_{\tiny{MLE}}(X; Y) + (K_X + K_Y - K_{XY} - 1)/(2N \cdot \log(\texttt{base}))\\ \end{align}\end{split}\]

For an arbitrary number of random variables this is equivalent to:

\[\begin{align}\hat{I}_{\tiny{MM}}(X_1; \dots; X_n) &= \hat{I}_{\tiny{MLE}}(X_1; \dots; X_n) + \frac{\left(\sum_{i=1}^{n}K_i-1\right) - \left(K_{1,\dots,i}-1\right)}{2N \cdot \log(\texttt{base})} \end{align}\]

\(\hat{I}_{\tiny{MLE}}(X_1; \dots; X_n)\) is the initial DiscreteMIEstimator estimate, \(K_i\) is the number of unique values in the i-th variable, \(K_{1,\dots,i}\) is the number of unique joint values, and \(N\) is the number of samples.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

infomeasure.estimators.mutual_information.nsb module

Module for the Nsb mutual information estimator.

class infomeasure.estimators.mutual_information.nsb.BaseNsbMIEstimator(*data, cond=None, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Nsb mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.nsb.NsbCMIEstimator(*data, cond=None, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseNsbMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Nsb mutual information.

Nsb conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.nsb.NsbEntropyEstimator

Nsb entropy estimator.

Notes

This estimator uses the Nsb entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.nsb.NsbMIEstimator(*data, cond=None, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseNsbMIEstimator, MutualInformationEstimator

Estimator for the Nsb mutual information.

Nsb mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.nsb.NsbEntropyEstimator

Nsb entropy estimator.

Notes

This estimator uses the Nsb entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

infomeasure.estimators.mutual_information.ordinal module

Module for the Ordinal / Permutation mutual information estimator.

class infomeasure.estimators.mutual_information.ordinal.BaseOrdinalMIEstimator(*data, cond=None, embedding_dim: int = None, step_size: int = 1, stable: bool = False, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for the Ordinal mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

embedding_dimint

The size of the permutation patterns.

step_sizeint, optional

Step size between elements for the state space reconstruction (delay).

stablebool, optional

If True, when sorting the data, the order of equal elements is preserved. This can be useful for reproducibility and testing, but might be slower.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

TypeError

If the data are not 1d array-like(s).

Notes

• The ordinality will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.ordinal.OrdinalCMIEstimator(*data, cond=None, embedding_dim: int = None, step_size: int = 1, stable: bool = False, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseOrdinalMIEstimator, ConditionalMutualInformationEstimator

Estimator for the Ordinal conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

embedding_dimint

The size of the permutation patterns.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

symbols_condarray_like, shape (n_samples,)

The symbolized conditional data used to estimate the conditional mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The order will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.ordinal.OrdinalMIEstimator(*data, cond=None, embedding_dim: int = None, step_size: int = 1, stable: bool = False, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseOrdinalMIEstimator, MutualInformationEstimator

Estimator for the Ordinal mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

embedding_dimint

The size of the permutation patterns.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

Raises:

ValueError

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The ordinality will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

infomeasure.estimators.mutual_information.renyi module

Module for the Renyi mutual information estimator.

class infomeasure.estimators.mutual_information.renyi.BaseRenyiMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Renyi mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

class infomeasure.estimators.mutual_information.renyi.RenyiCMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseRenyiMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

class infomeasure.estimators.mutual_information.renyi.RenyiMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseRenyiMIEstimator, MutualInformationEstimator

Estimator for the Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

infomeasure.estimators.mutual_information.shrink module

Module for the Shrink mutual information estimator.

class infomeasure.estimators.mutual_information.shrink.BaseShrinkMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Shrink mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.shrink.ShrinkCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseShrinkMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Shrink mutual information.

Shrink conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.shrink.ShrinkEntropyEstimator

Shrink entropy estimator.

Notes

This estimator uses the Shrink entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.shrink.ShrinkMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseShrinkMIEstimator, MutualInformationEstimator

Estimator for the Shrink mutual information.

Shrink mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.shrink.ShrinkEntropyEstimator

Shrink entropy estimator.

Notes

This estimator uses the Shrink entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

infomeasure.estimators.mutual_information.tsallis module

Module for the Tsallis mutual information estimator.

class infomeasure.estimators.mutual_information.tsallis.BaseTsallisMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Tsallis mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

class infomeasure.estimators.mutual_information.tsallis.TsallisCMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseTsallisMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Tsallis mutual information.

Parameters:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

qfloat | int

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

normalize

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

class infomeasure.estimators.mutual_information.tsallis.TsallisMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseTsallisMIEstimator, MutualInformationEstimator

Estimator for the Tsallis mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

infomeasure.estimators.mutual_information.zhang module

Module for the Zhang mutual information estimator.

class infomeasure.estimators.mutual_information.zhang.BaseZhangMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: ABC

Base class for Zhang mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.zhang.ZhangCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseZhangMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Zhang mutual information.

Zhang conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.zhang.ZhangEntropyEstimator

Zhang entropy estimator.

Notes

This estimator uses the Zhang entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.zhang.ZhangMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseZhangMIEstimator, MutualInformationEstimator

Estimator for the Zhang mutual information.

Zhang mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.zhang.ZhangEntropyEstimator

Zhang entropy estimator.

Notes

This estimator uses the Zhang entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

Module contents

Mutual information estimators.

class infomeasure.estimators.mutual_information.AnsbCMIEstimator(*data, cond=None, undersampled: float = 0.1, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseAnsbMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Ansb mutual information.

Ansb conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

undersampledfloat, default=0.1

Maximum allowed ratio N/K to consider data sufficiently undersampled. A warning is issued if this threshold is exceeded.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.ansb.AnsbEntropyEstimator

Ansb entropy estimator.

Notes

This estimator uses the Ansb entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.AnsbMIEstimator(*data, cond=None, undersampled: float = 0.1, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseAnsbMIEstimator, MutualInformationEstimator

Estimator for the Ansb mutual information.

Ansb mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

undersampledfloat, default=0.1

Maximum allowed ratio N/K to consider data sufficiently undersampled. A warning is issued if this threshold is exceeded.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.ansb.AnsbEntropyEstimator

Ansb entropy estimator.

Notes

This estimator uses the Ansb entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.BayesCMIEstimator(*data, cond=None, alpha: float | str, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBayesMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Bayes mutual information.

Bayesian conditional mutual information estimator using Dirichlet prior with concentration parameter α. Provides principled handling of sparse data through Bayesian probability estimates.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

alphafloat | str

The concentration parameter α of the Dirichlet prior. Either a float or a string specifying the choice of concentration parameter.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bayes.BayesEntropyEstimator

Bayesian entropy estimator with Dirichlet prior.

Notes

This estimator uses Bayesian probability estimates with a Dirichlet prior to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.BayesMIEstimator(*data, cond=None, alpha: float | str, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBayesMIEstimator, MutualInformationEstimator

Estimator for the Bayes mutual information.

Bayesian mutual information estimator using Dirichlet prior with concentration parameter α. Provides principled handling of sparse data through Bayesian probability estimates.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

alphafloat | str

The concentration parameter α of the Dirichlet prior. Either a float or a string specifying the choice of concentration parameter.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bayes.BayesEntropyEstimator

Bayesian entropy estimator with Dirichlet prior.

Notes

This estimator uses Bayesian probability estimates with a Dirichlet prior to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.BonachelaCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBonachelaMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Bonachela mutual information.

Bonachela conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bonachela.BonachelaEntropyEstimator

Bonachela entropy estimator.

Notes

This estimator uses the Bonachela entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.BonachelaMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseBonachelaMIEstimator, MutualInformationEstimator

Estimator for the Bonachela mutual information.

Bonachela mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.bonachela.BonachelaEntropyEstimator

Bonachela entropy estimator.

Notes

This estimator uses the Bonachela entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ChaoShenCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoShenMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional ChaoShen mutual information.

ChaoShen conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_shen.ChaoShenEntropyEstimator

ChaoShen entropy estimator.

Notes

This estimator uses the ChaoShen entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ChaoShenMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoShenMIEstimator, MutualInformationEstimator

Estimator for the ChaoShen mutual information.

ChaoShen mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_shen.ChaoShenEntropyEstimator

ChaoShen entropy estimator.

Notes

This estimator uses the ChaoShen entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ChaoWangJostCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoWangJostMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional ChaoWangJost mutual information.

ChaoWangJost conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_wang_jost.ChaoWangJostEntropyEstimator

ChaoWangJost entropy estimator.

Notes

This estimator uses the ChaoWangJost entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ChaoWangJostMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseChaoWangJostMIEstimator, MutualInformationEstimator

Estimator for the ChaoWangJost mutual information.

ChaoWangJost mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.chao_wang_jost.ChaoWangJostEntropyEstimator

ChaoWangJost entropy estimator.

Notes

This estimator uses the ChaoWangJost entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.DiscreteCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseDiscreteMIEstimator, ConditionalMutualInformationEstimator

Estimator for the discrete conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

class infomeasure.estimators.mutual_information.DiscreteMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseDiscreteMIEstimator, MutualInformationEstimator

Estimator for the discrete mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

class infomeasure.estimators.mutual_information.GrassbergerCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseGrassbergerMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Grassberger mutual information.

Grassberger conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.grassberger.GrassbergerEntropyEstimator

Grassberger entropy estimator.

Notes

This estimator uses the Grassberger entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.GrassbergerMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseGrassbergerMIEstimator, MutualInformationEstimator

Estimator for the Grassberger mutual information.

Grassberger mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.grassberger.GrassbergerEntropyEstimator

Grassberger entropy estimator.

Notes

This estimator uses the Grassberger entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.KSGCMIEstimator(*data, cond=None, k: int = 4, ksg_id: int = 1, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKSGMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

normalizebool, optional

If True, normalize the data before analysis.

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KSG11].

class infomeasure.estimators.mutual_information.KSGMIEstimator(*data, cond=None, k: int = 4, ksg_id: int = 1, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKSGMIEstimator, MutualInformationEstimator

Estimator for mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The estimator supports two variants:

• Type I (ksg_id=1): Uses strict inequality for counting neighbors in marginal spaces (dist < eps).

• Type II (ksg_id=2): Uses non-strict inequality (dist <= eps) and a modified formula.

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KSG11].

class infomeasure.estimators.mutual_information.KernelCMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKernelMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using Kernel Density Estimation (KDE).

\[I(X;Y|Z) = \sum_{i=1}^{n} p(x_i, y_i, z_i) \log \left( \frac{p(z_i)p(x_i, y_i, z_i)}{p(x_i, z_i)p(y_i, z_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

class infomeasure.estimators.mutual_information.KernelMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseKernelMIEstimator, MutualInformationEstimator

Estimator for mutual information using Kernel Density Estimation (KDE).

\[I(X;Y) = \sum_{i=1}^{n} p(x_i, y_i) \log \left( \frac{p(x_i, y_i)}{p(x_i)p(y_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

class infomeasure.estimators.mutual_information.MillerMadowCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseMillerMadowMIEstimator, ConditionalMutualInformationEstimator

Estimator for the discrete Miller-Madow conditional mutual information.

\[\begin{align} \hat{I}_{\tiny{MM}}(X_1; X_2; \ldots; X_n \mid Z)&= \hat{I}_{\tiny{MLE}}(\dots) + \frac{\left(\sum_{i=1}^{n}K_{iZ}-1\right) - \left(K_{1,\dots,i,Z}-1\right) - \left(K_{Z}-1\right)}{2N \cdot \log(\texttt{base})} \end{align}\]

\(\hat{I}_{\tiny{MLE}}(X_1; \dots; X_n \mid Z)\) is the initial DiscreteCMIEstimator estimate, \(K_{iZ}\) is the number of unique values in the i-th variable joint with Z, \(K_{1,\dots,i,Z}\) is the number of unique joint values, \(K_Z\) is the number of unique values in the Z variable, and \(N\) is the number of samples.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

class infomeasure.estimators.mutual_information.MillerMadowMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseMillerMadowMIEstimator, MutualInformationEstimator

Estimator for the discrete Miller-Madow mutual information.

\[\begin{split}\begin{align}\hat{I}_{\tiny{MM}}(X; Y) &= \hat{H}_{\tiny{MM}}(X) + \hat{H}_{\tiny{MM}}(Y) - \hat{H}_{\tiny{MM}}(X, Y)\\ &= \hat{H}_{\tiny{MLE}}(X) + \hat{H}_{\tiny{MLE}}(Y) - \hat{H}_{\tiny{MLE}}(X, Y) + (K_X + K_Y - K_{XY} - 1)/(2N \cdot \log(\texttt{base}))\\ &= \hat{I}_{\tiny{MLE}}(X; Y) + (K_X + K_Y - K_{XY} - 1)/(2N \cdot \log(\texttt{base}))\\ \end{align}\end{split}\]

For an arbitrary number of random variables this is equivalent to:

\[\begin{align}\hat{I}_{\tiny{MM}}(X_1; \dots; X_n) &= \hat{I}_{\tiny{MLE}}(X_1; \dots; X_n) + \frac{\left(\sum_{i=1}^{n}K_i-1\right) - \left(K_{1,\dots,i}-1\right)}{2N \cdot \log(\texttt{base})} \end{align}\]

\(\hat{I}_{\tiny{MLE}}(X_1; \dots; X_n)\) is the initial DiscreteMIEstimator estimate, \(K_i\) is the number of unique values in the i-th variable, \(K_{1,\dots,i}\) is the number of unique joint values, and \(N\) is the number of samples.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

class infomeasure.estimators.mutual_information.NsbCMIEstimator(*data, cond=None, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseNsbMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Nsb mutual information.

Nsb conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.nsb.NsbEntropyEstimator

Nsb entropy estimator.

Notes

This estimator uses the Nsb entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.NsbMIEstimator(*data, cond=None, K: int = None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseNsbMIEstimator, MutualInformationEstimator

Estimator for the Nsb mutual information.

Nsb mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

Kint, optional

The support size. If not provided, uses the observed support size.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.nsb.NsbEntropyEstimator

Nsb entropy estimator.

Notes

This estimator uses the Nsb entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.OrdinalCMIEstimator(*data, cond=None, embedding_dim: int = None, step_size: int = 1, stable: bool = False, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseOrdinalMIEstimator, ConditionalMutualInformationEstimator

Estimator for the Ordinal conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

embedding_dimint

The size of the permutation patterns.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

symbols_condarray_like, shape (n_samples,)

The symbolized conditional data used to estimate the conditional mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The order will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.OrdinalMIEstimator(*data, cond=None, embedding_dim: int = None, step_size: int = 1, stable: bool = False, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseOrdinalMIEstimator, MutualInformationEstimator

Estimator for the Ordinal mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

embedding_dimint

The size of the permutation patterns.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

Raises:

ValueError

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The ordinality will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.RenyiCMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseRenyiMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

class infomeasure.estimators.mutual_information.RenyiMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseRenyiMIEstimator, MutualInformationEstimator

Estimator for the Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

class infomeasure.estimators.mutual_information.ShrinkCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseShrinkMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Shrink mutual information.

Shrink conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.shrink.ShrinkEntropyEstimator

Shrink entropy estimator.

Notes

This estimator uses the Shrink entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ShrinkMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseShrinkMIEstimator, MutualInformationEstimator

Estimator for the Shrink mutual information.

Shrink mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.shrink.ShrinkEntropyEstimator

Shrink entropy estimator.

Notes

This estimator uses the Shrink entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.TsallisCMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseTsallisMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Tsallis mutual information.

Parameters:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

qfloat | int

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

normalize

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

class infomeasure.estimators.mutual_information.TsallisMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e', **kwargs)

Bases: BaseTsallisMIEstimator, MutualInformationEstimator

Estimator for the Tsallis mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

class infomeasure.estimators.mutual_information.ZhangCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseZhangMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Zhang mutual information.

Zhang conditional mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.zhang.ZhangEntropyEstimator

Zhang entropy estimator.

Notes

This estimator uses the Zhang entropy estimator to compute conditional mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_cmi_from_entropy) not a dedicated implementation as other MI might have.

class infomeasure.estimators.mutual_information.ZhangMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e', **kwargs)

Bases: BaseZhangMIEstimator, MutualInformationEstimator

Estimator for the Zhang mutual information.

Zhang mutual information estimator using the entropy combination formula.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

See also

infomeasure.estimators.entropy.zhang.ZhangEntropyEstimator

Zhang entropy estimator.

Notes

This estimator uses the Zhang entropy estimator to compute mutual information through the entropy combination formula.

Note that the entropy combination formula is used (_generic_mi_from_entropy) not a dedicated implementation as other MI might have.