entropy

Contents

entropy#

infomeasure.entropy(*data, approach: str, **kwargs: any)[source]#

Calculate the (joint) entropy using a functional interface of different estimators.

Supports the following approaches:

  1. ansb: Asymptotic NSB entropy estimator.

  2. bayes: Bayesian entropy estimator.

  3. bonachela: Bonachela entropy estimator.

  4. [chao_shen, cs]: Chao-Shen entropy estimator.

  5. [chao_wang_jost, cwj]: Chao Wang Jost entropy estimator.

  6. discrete: Discrete entropy estimator.

  7. grassberger: Grassberger entropy estimator.

  8. kernel: Kernel entropy estimator.

  9. [metric, kl]: Kozachenko-Leonenko entropy estimator.

  10. [miller_madow, mm]: Miller-Madow entropy estimator.

  11. nsb: NSB (Nemenman-Shafee-Bialek) entropy estimator.

  12. [ordinal, symbolic, permutation]: Ordinal / Permutation entropy estimator.

  13. renyi: Renyi entropy estimator.

  14. [shrink, js]: Shrinkage (James-Stein) entropy estimator.

  15. tsallis: Tsallis entropy estimator.

  16. zhang: Zhang entropy estimator.

For the discrete Shannon entropy this is

\[\texttt{im.entropy(data_X, approach="discrete")} = H(X) = -\sum_{x \in X} p(x) \log p(x).\]

Where for \(H(x)\), the estimated pmf \(p(x)\) belongs to the RV \(X\).

\[\texttt{im.entropy(data_P, data_Q, ...)} = H_Q(P) = H_\times(P, Q) = -\sum_{x \in X} p(x) \log q(x)\]

For the cross-entropy \(H_Q(P)\), the estimated pmf \(p(x)\) belongs to the RV \(P\) and \(q(x)\) to the RV \(Q\). For other approaches, this formula is generalized in different forms.

Parameters:
*dataarray_like

The data used to estimate the entropy. For entropy, this can be an array-like. For joint entropy, pass the joint values inside a tuple. For cross-entropy, pass two separate parameters.

approachstr

The name of the estimator to use.

**kwargs: dict

Additional keyword arguments to pass to the estimator.

Returns:
float

The calculated entropy.

Raises:
ValueError

If the estimator is not recognised.

Contents