infomeasure.estimators.entropy package

Submodules

infomeasure.estimators.entropy.discrete module

Module for the discrete entropy estimator.

class infomeasure.estimators.entropy.discrete.DiscreteEntropyEstimator(data, *, base: int | float | str = 'e')

Bases: DistributionMixin, EntropyEstimator

Estimator for discrete entropy (Shannon entropy).

Attributes:

dataarray_like

The data used to estimate the entropy.

infomeasure.estimators.entropy.kernel module

Module for the kernel entropy estimator.

class infomeasure.estimators.entropy.kernel.KernelEntropyEstimator(data, *, bandwidth: float | int, kernel: str, workers: int = 1, base: int | float | str = 'e')

Bases: WorkersMixin, EntropyEstimator

Estimator for entropy (Shannon) using Kernel Density Estimation (KDE).

Attributes:

dataarray_like

The data used to estimate the entropy.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

workersint, optional

Number of workers to use for parallel processing. Default is 1, meaning no parallel processing. If set to -1, all available CPU cores will be used.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

infomeasure.estimators.entropy.kozachenko_leonenko module

Module for the Kozacenko-Leonenko entropy estimator.

class infomeasure.estimators.entropy.kozachenko_leonenko.KozachenkoLeonenkoEntropyEstimator(data, *, k: int = 4, noise_level=1e-10, minkowski_p=inf, base: int | float | str = 'e')

Bases: RandomGeneratorMixin, EntropyEstimator

Kozachenko-Leonenko estimator for Shannon entropies.

Attributes:

dataarray_like

The data used to estimate the entropy.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

Raises:

ValueError

If the number of nearest neighbors is not a positive integer

ValueError

If the noise level is negative

ValueError

If the Minkowski power parameter is invalid

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KStogbauerG11].

infomeasure.estimators.entropy.ordinal module

Module for the Ordinal / Permutation entropy estimator.

class infomeasure.estimators.entropy.ordinal.OrdinalEntropyEstimator(data, *, embedding_dim: int, stable: bool = False, base: int | float | str = 'e')

Bases: DistributionMixin, EntropyEstimator

Estimator for the Ordinal / Permutation entropy.

The Ordinal entropy is a measure of the complexity of a time series. The input data needs to be comparable, i.e., the data should be ordinal, as the relative frequencies are calculated. For a given embedding_dim (length of considered subsequences), all \(n!\) possible permutations are considered and their relative frequencies are calculated [BP02].

Embedding delay is not supported natively.

Attributes:

dataarray_like

The data used to estimate the entropy.

embedding_dimint

The size of the permutation patterns.

stablebool, optional

If True, when sorting the data, the embedding_dim of equal elements is preserved. This can be useful for reproducibility and testing, but might be slower.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If the embedding_dim is too large for the given data.

TypeError

If the data are not 1d array-like(s).

Notes

• The ordinality will be determined via numpy.argsort().

• If embedding_dim is set to 1, the entropy is always 0.

infomeasure.estimators.entropy.renyi module

Module for the Rényi entropy estimator.

class infomeasure.estimators.entropy.renyi.RenyiEntropyEstimator(data, *, k: int = 4, alpha: float | int = None, base: int | float | str = 'e')

Bases: EntropyEstimator

Estimator for the Rényi entropy.

Attributes:

dataarray_like

The data used to estimate the entropy.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

Raises:

ValueError

If the Renyi parameter is not a positive number.

ValueError

If the number of nearest neighbors is not a positive integer.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

infomeasure.estimators.entropy.tsallis module

Module for Tsallis entropy estimator.

class infomeasure.estimators.entropy.tsallis.TsallisEntropyEstimator(data, *, k: int = 4, q: float | int = None, base: int | float | str = 'e')

Bases: EntropyEstimator

Estimator for the Tsallis entropy.

Attributes:

dataarray_like

The data used to estimate the entropy.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

Raises:

ValueError

If the Tsallis parameter is not a positive number.

ValueError

If the number of nearest neighbors is not a positive integer.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.

Module contents

Entropy estimators.

class infomeasure.estimators.entropy.DiscreteEntropyEstimator(data, *, base: int | float | str = 'e')

Bases: DistributionMixin, EntropyEstimator

Estimator for discrete entropy (Shannon entropy).

Attributes:

dataarray_like

The data used to estimate the entropy.

class infomeasure.estimators.entropy.KernelEntropyEstimator(data, *, bandwidth: float | int, kernel: str, workers: int = 1, base: int | float | str = 'e')

Bases: WorkersMixin, EntropyEstimator

Estimator for entropy (Shannon) using Kernel Density Estimation (KDE).

Attributes:

dataarray_like

The data used to estimate the entropy.

bandwidthfloat | int

The bandwidth for the kernel.

kernelstr

Type of kernel to use, compatible with the KDE implementation kde_probability_density_function().

workersint, optional

Number of workers to use for parallel processing. Default is 1, meaning no parallel processing. If set to -1, all available CPU cores will be used.

Notes

A small bandwidth can lead to under-sampling, while a large bandwidth may over-smooth the data, obscuring details.

class infomeasure.estimators.entropy.KozachenkoLeonenkoEntropyEstimator(data, *, k: int = 4, noise_level=1e-10, minkowski_p=inf, base: int | float | str = 'e')

Bases: RandomGeneratorMixin, EntropyEstimator

Kozachenko-Leonenko estimator for Shannon entropies.

Attributes:

dataarray_like

The data used to estimate the entropy.

kint

The number of nearest neighbors to consider.

noise_levelfloat

The standard deviation of the Gaussian noise to add to the data to avoid issues with zero distances.

minkowski_pfloat, \(1 \leq p \leq \infty\)

The power parameter for the Minkowski metric. Default is np.inf for maximum norm. Use 2 for Euclidean distance.

Raises:

ValueError

If the number of nearest neighbors is not a positive integer

ValueError

If the noise level is negative

ValueError

If the Minkowski power parameter is invalid

Notes

Changing the number of nearest neighbors k can change the outcome, but the default value of \(k=4\) is recommended by [KStogbauerG11].

class infomeasure.estimators.entropy.OrdinalEntropyEstimator(data, *, embedding_dim: int, stable: bool = False, base: int | float | str = 'e')

Bases: DistributionMixin, EntropyEstimator

Estimator for the Ordinal / Permutation entropy.

The Ordinal entropy is a measure of the complexity of a time series. The input data needs to be comparable, i.e., the data should be ordinal, as the relative frequencies are calculated. For a given embedding_dim (length of considered subsequences), all \(n!\) possible permutations are considered and their relative frequencies are calculated [BP02].

Embedding delay is not supported natively.

Attributes:

dataarray_like

The data used to estimate the entropy.

embedding_dimint

The size of the permutation patterns.

stablebool, optional

If True, when sorting the data, the embedding_dim of equal elements is preserved. This can be useful for reproducibility and testing, but might be slower.

Raises:

ValueError

If the embedding_dim is negative or not an integer.

ValueError

If the embedding_dim is too large for the given data.

TypeError

If the data are not 1d array-like(s).

Notes

• The ordinality will be determined via numpy.argsort().

• If embedding_dim is set to 1, the entropy is always 0.

class infomeasure.estimators.entropy.RenyiEntropyEstimator(data, *, k: int = 4, alpha: float | int = None, base: int | float | str = 'e')

Bases: EntropyEstimator

Estimator for the Rényi entropy.

Attributes:

dataarray_like

The data used to estimate the entropy.

kint

The number of nearest neighbors used in the estimation.

alphafloat | int

The Rényi parameter, order or exponent. Sometimes denoted as \(\alpha\) or \(q\).

Raises:

ValueError

If the Renyi parameter is not a positive number.

ValueError

If the number of nearest neighbors is not a positive integer.

Notes

The Rényi entropy is a generalization of Shannon entropy, where the small values of probabilities are emphasized for \(\alpha < 1\), and higher probabilities are emphasized for \(\alpha > 1\). For \(\alpha = 1\), it reduces to Shannon entropy. The Rényi-Entropy class can be particularly interesting for systems where additivity (in Shannon sense) is not always preserved, especially in nonlinear complex systems, such as when dealing with long-range forces.

class infomeasure.estimators.entropy.TsallisEntropyEstimator(data, *, k: int = 4, q: float | int = None, base: int | float | str = 'e')

Bases: EntropyEstimator

Estimator for the Tsallis entropy.

Attributes:

dataarray_like

The data used to estimate the entropy.

kint

The number of nearest neighbors used in the estimation.

qfloat

The Tsallis parameter, order or exponent. Sometimes denoted as \(q\), analogous to the Rényi parameter \(\alpha\).

Raises:

ValueError

If the Tsallis parameter is not a positive number.

ValueError

If the number of nearest neighbors is not a positive integer.

Notes

In the \(q \to 1\) limit, the Jackson sum (q-additivity) reduces to ordinary summation, and the Tallis entropy reduces to Shannon Entropy. This class of entropy measure is in particularly useful in the study in connection with long-range correlated systems and with non-equilibrium phenomena.