entropy

Contents

entropy#

infomeasure.entropy(*data, approach: str, **kwargs: any)[source]#

Calculate the (joint) entropy using a functional interface of different estimators.

Supports the following approaches:

  1. discrete: Discrete entropy estimator.

  2. kernel: Kernel entropy estimator.

  3. [metric, kl]: Kozachenko-Leonenko entropy estimator.

  4. renyi: Renyi entropy estimator.

  5. [ordinal, symbolic, permutation]: Ordinal / Permutation entropy estimator.

  6. tsallis: Tsallis entropy estimator.

For the discrete Shannon entropy this is

\[\texttt{im.entropy(data_X, approach="discrete")} = H(X) = -\sum_{x \in X} p(x) \log p(x).\]

Where for \(H(x)\), the estimated pmf \(p(x)\) belongs to the RV \(X\).

\[\texttt{im.entropy(data_P, data_Q, ...)} = H_Q(P) = H_\times(P, Q) = -\sum_{x \in X} p(x) \log q(x)\]

For the cross-entropy \(H_Q(P)\), the estimated pmf \(p(x)\) belongs to the RV \(P\) and \(q(x)\) to the RV \(Q\). For other approaches, this formula is generalized in different forms.

Parameters:
*dataarray_like

The data used to estimate the entropy. For entropy, this can be an array-like. For joint entropy, pass the joint values inside a tuple. For cross-entropy, pass two separate parameters.

approachstr

The name of the estimator to use.

**kwargs: dict

Additional keyword arguments to pass to the estimator.

Returns:
float

The calculated entropy.

Raises:
ValueError

If the estimator is not recognized.

Contents