infomeasure.estimators.mutual_information package

Submodules

infomeasure.estimators.mutual_information.discrete module

Module for the discrete mutual information estimator.

class infomeasure.estimators.mutual_information.discrete.BaseDiscreteMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e')

Bases: ABC

Base class for discrete mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

class infomeasure.estimators.mutual_information.discrete.DiscreteCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e')

Bases: BaseDiscreteMIEstimator, ConditionalMutualInformationEstimator

Estimator for the discrete conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

class infomeasure.estimators.mutual_information.discrete.DiscreteMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e')

Bases: BaseDiscreteMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the discrete mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

infomeasure.estimators.mutual_information.kernel module

Module for the kernel-based mutual information estimator.

class infomeasure.estimators.mutual_information.kernel.BaseKernelMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e')

Bases: WorkersMixin, ABC

Base class for mutual information using Kernel Density Estimation (KDE).

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

workersint, optional

Number of workers to use for parallel processing. Default is 1, meaning no parallel processing. If set to -1, all available CPU cores will be used.

class infomeasure.estimators.mutual_information.kernel.KernelCMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKernelMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using Kernel Density Estimation (KDE).

\[I(X;Y|Z) = \sum_{i=1}^{n} p(x_i, y_i, z_i) \log \left( \frac{p(z_i)p(x_i, y_i, z_i)}{p(x_i, z_i)p(y_i, z_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

class infomeasure.estimators.mutual_information.kernel.KernelMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKernelMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for mutual information using Kernel Density Estimation (KDE).

\[I(X;Y) = \sum_{i=1}^{n} p(x_i, y_i) \log \left( \frac{p(x_i, y_i)}{p(x_i)p(y_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger module

Module for the Kraskov-Stoegbauer-Grassberger (KSG) mutual information estimator.

class infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger.BaseKSGMIEstimator(*data, cond=None, k: int = 4, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: ABC

Base class for mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

class infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger.KSGCMIEstimator(*data, cond=None, k: int = 4, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKSGMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

normalizebool, optional

If True, normalize the data before analysis.

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KStogbauerG11].

class infomeasure.estimators.mutual_information.kraskov_stoegbauer_grassberger.KSGMIEstimator(*data, cond=None, k: int = 4, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKSGMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KStogbauerG11].

infomeasure.estimators.mutual_information.ordinal module

Module for the Ordinal / Permutation mutual information estimator.

class infomeasure.estimators.mutual_information.ordinal.BaseOrdinalMIEstimator(*data, cond=None, embedding_dim: int = None, stable: bool = False, offset: int = 0, base: int | float | str = 'e')

Bases: ABC

Base class for the Ordinal mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

embedding_dimint

The size of the permutation patterns.

stablebool, optional

If True, when sorting the data, the order of equal elements is preserved. This can be useful for reproducibility and testing, but might be slower.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

TypeError

If the data are not 1d array-like(s).

Notes

• The ordinality will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.ordinal.OrdinalCMIEstimator(*args, **kwargs)

Bases: BaseOrdinalMIEstimator, ConditionalMutualInformationEstimator

Estimator for the Ordinal conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

embedding_dimint

The size of the permutation patterns.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

symbols_condarray_like, shape (n_samples,)

The symbolized conditional data used to estimate the conditional mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The order will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.ordinal.OrdinalMIEstimator(*data, cond=None, embedding_dim: int = None, stable: bool = False, offset: int = 0, base: int | float | str = 'e')

Bases: BaseOrdinalMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the Ordinal mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

embedding_dimint

The size of the permutation patterns.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The ordinality will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

infomeasure.estimators.mutual_information.renyi module

Module for the Renyi mutual information estimator.

class infomeasure.estimators.mutual_information.renyi.BaseRenyiMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: ABC

Base class for Renyi mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

class infomeasure.estimators.mutual_information.renyi.RenyiCMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseRenyiMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

class infomeasure.estimators.mutual_information.renyi.RenyiMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseRenyiMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

infomeasure.estimators.mutual_information.tsallis module

Module for the Tsallis mutual information estimator.

class infomeasure.estimators.mutual_information.tsallis.BaseTsallisMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: ABC

Base class for Tsallis mutual information estimators.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the (conditional) mutual information. You can pass an arbitrary number of data arrays as positional arguments. For conditional mutual information, only two data arrays are allowed.

condarray_like, optional

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift. Not compatible with the cond parameter / conditional MI.

normalizebool, optional

If True, normalize the data before analysis.

class infomeasure.estimators.mutual_information.tsallis.TsallisCMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseTsallisMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Tsallis mutual information.

Parameters:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

qfloat | int

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

normalize

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

class infomeasure.estimators.mutual_information.tsallis.TsallisMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseTsallisMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the Tsallis mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

Module contents

Mutual information estimators.

class infomeasure.estimators.mutual_information.DiscreteCMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e')

Bases: BaseDiscreteMIEstimator, ConditionalMutualInformationEstimator

Estimator for the discrete conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

class infomeasure.estimators.mutual_information.DiscreteMIEstimator(*data, cond=None, offset: int = 0, base: int | float | str = 'e')

Bases: BaseDiscreteMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the discrete mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

class infomeasure.estimators.mutual_information.KSGCMIEstimator(*data, cond=None, k: int = 4, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKSGMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

normalizebool, optional

If True, normalize the data before analysis.

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KStogbauerG11].

class infomeasure.estimators.mutual_information.KSGMIEstimator(*data, cond=None, k: int = 4, noise_level=1e-10, minkowski_p=inf, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKSGMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for mutual information using the Kraskov-Stoegbauer-Grassberger (KSG) method.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KStogbauerG11].

class infomeasure.estimators.mutual_information.KernelCMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKernelMIEstimator, ConditionalMutualInformationEstimator

Estimator for conditional mutual information using Kernel Density Estimation (KDE).

\[I(X;Y|Z) = \sum_{i=1}^{n} p(x_i, y_i, z_i) \log \left( \frac{p(z_i)p(x_i, y_i, z_i)}{p(x_i, z_i)p(y_i, z_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

class infomeasure.estimators.mutual_information.KernelMIEstimator(*data, cond=None, bandwidth: float | int = None, kernel: str = None, offset: int = 0, workers: int = 1, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseKernelMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for mutual information using Kernel Density Estimation (KDE).

\[I(X;Y) = \sum_{i=1}^{n} p(x_i, y_i) \log \left( \frac{p(x_i, y_i)}{p(x_i)p(y_i)} \right)\]

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

class infomeasure.estimators.mutual_information.OrdinalCMIEstimator(*args, **kwargs)

Bases: BaseOrdinalMIEstimator, ConditionalMutualInformationEstimator

Estimator for the Ordinal conditional mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

embedding_dimint

The size of the permutation patterns.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

symbols_condarray_like, shape (n_samples,)

The symbolized conditional data used to estimate the conditional mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The order will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.OrdinalMIEstimator(*data, cond=None, embedding_dim: int = None, stable: bool = False, offset: int = 0, base: int | float | str = 'e')

Bases: BaseOrdinalMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the Ordinal mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

embedding_dimint

The size of the permutation patterns.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

*symbolsarray_like, shape (n_samples,)

The symbolized data used to estimate the mutual information.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If offset and embedding_dim are such that the data is too small.

Notes

• The ordinality will be determined via numpy.argsort(). There is no normalize option, as this would not influence the order of the data.

• If embedding_dim is set to 1, the mutual information is always 0.

class infomeasure.estimators.mutual_information.RenyiCMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseRenyiMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

class infomeasure.estimators.mutual_information.RenyiMIEstimator(*data, cond=None, k: int = 4, alpha: float | int = None, noise_level=1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseRenyiMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the Renyi mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

class infomeasure.estimators.mutual_information.TsallisCMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseTsallisMIEstimator, ConditionalMutualInformationEstimator

Estimator for the conditional Tsallis mutual information.

Parameters:

*dataarray_like, shape (n_samples,)

The data used to estimate the conditional mutual information. You can pass an arbitrary number of data arrays as positional arguments.

condarray_like

The conditional data used to estimate the conditional mutual information.

kint

The number of nearest neighbors to consider.

qfloat | int

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

normalize

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

class infomeasure.estimators.mutual_information.TsallisMIEstimator(*data, cond=None, k: int = 4, q: float | int = None, noise_level: float = 1e-08, offset: int = 0, normalize: bool = False, base: int | float | str = 'e')

Bases: BaseTsallisMIEstimator, PValueMixin, MutualInformationEstimator

Estimator for the Tsallis mutual information.

Attributes:

*dataarray_like, shape (n_samples,)

The data used to estimate the mutual information. You can pass an arbitrary number of data arrays as positional arguments.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

offsetint, optional

Number of positions to shift the data arrays relative to each other. Delay/lag/shift between the variables. Default is no shift.

normalizebool, optional

If True, normalize the data before analysis.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.